2020-11-16T18:28:26.500 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-16T18:28:26.501 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --block --verbose --archive /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024 --name teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi --description powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} --lock -- /tmp/teuthology-worker.9hgetleg.tmp 2020-11-16T18:28:26.527 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T18:28:26.578 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024 branch: master description: powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} email: ceph-qa@ceph.io first_in_suite: false job_id: '5631024' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi nuke-on-error: true os_type: centos os_version: '8.1' overrides: admin_socket: branch: master ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - \(MDS_TRIM\) - \(MDS_SLOW_REQUEST\) - MDS_SLOW_METADATA_IO - Behind on trimming log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b637909a9153a419511f943e4eee7800ad3bbb99 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: b637909a9153a419511f943e4eee7800ad3bbb99 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: master sha1: 6441771be52a0f47b9e118f8bca335f29dfda438 owner: scheduled_teuthology@teuthology priority: 80 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - client.0 - - osd.0 - - osd.1 - - osd.2 sha1: b637909a9153a419511f943e4eee7800ad3bbb99 sleep_before_teardown: 0 suite: powercycle suite_branch: master suite_path: /home/teuthworker/src/git.ceph.com_ceph_master/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 6441771be52a0f47b9e118f8bca335f29dfda438 tasks: - install: extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - thrashosds: chance_down: 1.0 powercycle: true timeout: 600 - ceph-fuse: null - workunit: clients: all: - suites/fsx.sh timeout: 6h teuthology_branch: master timestamp: 2020-11-16_11:15:03 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.11568 2020-11-16T18:28:26.579 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_master/qa; will attempt to use it 2020-11-16T18:28:26.579 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_master/qa/tasks 2020-11-16T18:28:26.580 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-16T18:28:26.581 INFO:teuthology.task.internal:Checking packages... 2020-11-16T18:28:26.601 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'b637909a9153a419511f943e4eee7800ad3bbb99' 2020-11-16T18:28:26.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-16T18:28:26.602 INFO:teuthology.packaging:ref: None 2020-11-16T18:28:26.602 INFO:teuthology.packaging:tag: None 2020-11-16T18:28:26.602 INFO:teuthology.packaging:branch: master 2020-11-16T18:28:26.603 INFO:teuthology.packaging:sha1: b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T18:28:26.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=master 2020-11-16T18:28:27.401 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7207.gc10a7240 2020-11-16T18:28:27.402 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-16T18:28:27.411 INFO:teuthology.task.internal:no buildpackages task found 2020-11-16T18:28:27.411 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-16T18:28:28.056 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-16T18:28:28.057 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T18:28:28.107 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:28:38.124 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:28:48.134 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:28:58.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:08.165 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:18.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:28.199 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:38.214 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:48.232 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:29:58.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:08.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:18.282 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:28.299 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:38.316 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:48.325 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:30:58.337 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:31:08.355 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:31:18.371 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:31:28.387 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:31:38.401 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:31:48.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:31:58.434 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:32:08.451 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:32:18.470 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:32:28.487 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:32:38.505 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:32:48.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:32:58.539 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:33:08.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:33:18.576 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:33:28.591 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:33:38.608 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:33:48.625 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:33:58.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:34:08.659 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:34:18.675 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:34:28.693 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:34:38.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:34:48.731 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:34:58.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:35:08.762 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:35:18.775 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:35:28.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:35:38.807 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:35:48.823 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:35:58.841 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:36:08.859 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:36:18.880 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:36:28.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T18:36:38.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:36:48.926 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:36:58.940 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:37:08.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:37:18.978 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:37:28.995 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T18:37:39.020 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:37:49.039 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T18:38:21.754 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:38:31.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:38:41.788 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:38:51.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:01.822 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:11.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:21.857 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:31.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:41.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:39:51.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:01.931 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:11.940 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:21.958 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:31.976 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:41.991 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:40:52.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:02.028 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:12.043 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:22.056 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:32.075 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:42.097 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:41:52.108 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:02.122 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:12.134 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:22.148 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:32.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:42.181 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:42:52.199 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:02.214 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:12.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:22.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:32.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:42.279 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:43:52.298 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:02.308 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:12.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:22.345 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:32.361 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:42.382 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:44:52.400 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:02.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:12.428 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:22.448 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:32.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:42.485 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:45:52.502 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:02.518 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:12.538 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:22.558 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:32.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:42.599 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:46:52.613 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:02.632 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:12.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:22.666 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:32.685 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:42.704 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:47:52.721 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:02.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:12.758 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:22.771 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:32.791 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:42.810 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:48:52.825 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:02.842 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:12.862 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:22.881 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:32.898 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:42.915 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:49:52.933 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:02.952 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:12.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:22.993 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:33.011 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:43.032 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:50:53.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:03.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:13.083 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:23.101 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:33.119 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:43.140 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:51:53.158 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:03.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:13.197 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:23.214 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:33.234 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:43.254 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:52:53.273 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:03.289 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:13.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:23.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:33.342 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:43.361 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:53:53.376 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:03.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:13.411 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:23.430 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:33.449 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:43.468 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:54:53.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:03.491 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:13.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:23.530 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:33.549 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:43.567 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:55:53.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:03.604 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:13.626 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:23.644 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:33.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:43.684 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:56:53.703 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:57:03.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T18:57:13.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:22:23.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:22:33.714 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:22:43.728 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:22:53.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:23:03.762 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:23:13.781 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:23:23.797 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:23:33.815 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:23:43.830 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:23:53.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T19:24:03.853 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T19:24:13.868 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:24:23.886 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:24:33.899 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:24:43.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:24:53.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:03.943 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:13.961 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:23.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:33.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:44.005 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:25:54.022 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:04.039 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:14.050 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:24.065 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:34.082 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:44.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:26:54.116 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:27:04.133 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:27:14.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:27:24.160 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:27:34.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:07.509 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:17.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:27.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:37.558 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:47.576 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:28:57.595 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:07.615 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:17.631 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:27.649 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:37.668 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:47.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:29:57.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:07.720 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:17.744 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:27.762 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:37.779 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:47.796 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:30:57.811 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:07.826 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:17.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:27.863 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:37.882 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:47.896 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:31:57.912 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:07.931 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:17.950 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:27.968 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:37.986 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:48.002 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:32:58.015 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:08.030 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:18.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:28.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:38.088 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:48.103 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:33:58.113 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:34:08.122 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T19:34:18.140 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:34:28.159 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:34:38.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:34:48.196 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:34:58.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:35:08.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:35:18.245 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:35:28.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:35:38.279 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:35:48.293 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:35:58.310 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:08.329 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:18.347 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:28.365 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:38.378 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:48.393 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:36:58.411 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:37:08.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:37:18.461 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:37:28.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:37:38.495 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:37:48.510 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:37:58.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:38:08.544 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:38:18.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:38:28.571 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:38:38.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:38:48.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:38:58.633 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:08.652 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:18.669 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:28.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:38.696 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:48.708 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:39:58.727 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:08.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:18.752 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:28.769 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:38.787 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:48.797 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:40:58.816 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:41:08.826 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:41:18.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:41:28.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:41:38.878 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:41:48.896 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:41:58.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:08.929 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:18.948 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:28.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:38.978 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:48.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:42:59.015 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:43:09.031 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T19:43:19.041 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:43:29.059 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:43:39.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:43:49.093 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:43:59.113 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:09.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:19.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:29.165 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:39.175 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:49.198 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:44:59.215 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:45:09.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:45:19.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:45:29.272 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:45:39.287 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:45:49.305 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:45:59.323 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:46:09.334 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:46:19.352 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:46:29.366 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:46:39.378 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:46:49.397 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:46:59.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:09.431 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:19.448 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:29.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:39.485 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:49.501 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:47:59.521 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:48:09.540 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T19:48:19.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:48:29.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:48:39.595 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:48:49.606 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:48:59.627 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:09.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:19.664 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:29.674 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:39.691 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:49.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:49:59.724 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:09.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:19.756 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:29.775 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:39.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:49.805 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:50:59.817 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T19:51:09.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:51:19.846 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:51:29.864 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:51:39.882 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:51:49.894 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:51:59.910 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:52:09.929 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:52:19.947 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:52:29.958 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:52:39.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:52:49.992 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:00.011 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:10.027 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:20.043 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:30.062 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:40.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:53:50.091 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:00.110 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:10.120 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:20.139 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:30.156 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:40.175 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:54:50.192 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:00.210 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:10.229 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:20.243 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:30.262 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:40.282 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:55:50.297 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:00.312 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:10.324 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:20.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:30.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:40.372 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:56:50.392 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:00.406 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:10.422 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:20.439 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:30.450 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:40.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:57:50.481 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:00.500 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:10.519 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:20.528 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:30.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:40.563 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:58:50.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:00.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:10.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:20.631 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:30.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:40.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T19:59:50.679 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:00.700 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:10.716 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:20.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:30.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:40.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:00:50.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:01:00.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:01:10.812 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:01:20.828 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:01:30.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:01:40.863 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:01:50.881 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:00.898 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:10.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:20.934 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:30.951 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:40.965 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:02:50.976 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:00.991 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:11.006 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:21.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:31.032 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:41.045 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:03:51.064 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:01.078 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:11.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:21.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:31.127 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:41.142 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:04:51.160 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:01.172 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:11.187 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:21.206 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:31.225 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:41.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:05:51.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:01.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:11.288 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:21.347 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:31.363 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:41.378 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:06:51.395 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:01.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:11.432 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:21.451 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:31.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:41.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:07:51.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:08:01.530 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:08:11.539 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:08:21.556 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:08:31.571 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:08:41.590 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T20:08:51.603 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:01.622 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:11.634 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:21.652 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:31.668 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:41.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:09:51.703 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:01.722 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:11.740 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:21.758 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:31.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:41.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:10:51.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:01.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:11.846 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:21.858 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:31.869 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:41.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:11:51.906 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T20:12:01.925 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:12:11.941 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:12:21.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:12:31.977 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:12:41.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:12:52.013 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:02.031 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:12.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:22.061 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:32.077 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:42.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:13:52.111 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:02.126 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:12.145 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:22.161 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:32.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:42.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:14:52.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:15:02.218 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:15:12.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:15:22.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:15:32.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:15:42.297 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:15:52.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:02.338 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:12.350 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:22.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:32.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:42.398 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:16:52.414 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:17:02.432 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:17:12.451 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:17:22.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:17:32.483 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:17:42.499 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:17:52.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:02.530 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:12.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:22.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:32.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:42.596 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:18:52.609 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:19:02.627 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:20:19.307 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:20:29.325 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:20:39.344 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:20:49.358 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:20:59.374 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:09.391 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:19.405 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:29.417 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:39.434 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:49.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:21:59.468 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:09.486 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:19.503 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:29.523 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:39.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:49.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:22:59.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:09.592 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:19.611 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:29.630 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:39.648 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:49.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:23:59.679 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:09.695 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:19.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:29.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:39.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:49.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:24:59.775 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:09.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:19.817 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:29.836 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:39.854 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:49.873 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:25:59.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:26:09.909 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:26:19.928 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:26:29.943 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:26:39.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:26:49.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:26:59.985 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:27:10.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:27:20.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:27:30.030 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:27:40.049 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:27:50.066 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:00.076 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:10.093 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:20.111 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:30.129 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:40.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:28:50.160 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:00.180 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:10.190 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:20.204 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:30.221 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:40.238 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:29:50.247 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:00.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:10.272 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:20.283 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:30.298 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:40.313 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:30:50.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:00.363 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:10.380 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:20.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:30.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:40.433 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:31:50.446 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:00.464 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:10.481 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:20.499 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:30.518 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:40.536 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:32:50.549 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:00.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:10.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:20.594 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:30.612 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:40.630 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:33:50.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:34:00.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:34:10.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:34:20.689 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:34:30.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:34:40.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:34:50.744 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:35:00.763 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:35:10.781 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:35:20.799 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:35:30.818 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:35:40.836 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:35:50.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:01.366 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:11.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:21.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:31.410 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:41.427 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:36:51.437 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:37:01.448 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:37:11.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:37:21.482 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:37:31.501 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:37:41.510 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:37:51.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:01.546 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:11.556 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:21.566 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:31.582 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:41.598 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:38:51.617 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:01.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:11.653 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:21.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:31.689 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:41.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:39:51.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:40:10.656 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:40:20.675 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:40:30.692 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:40:40.711 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:40:50.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:00.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:10.757 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:20.776 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:30.793 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:40.811 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:41:50.828 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:00.848 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:10.859 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:20.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:30.886 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:40.897 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:42:50.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:00.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:10.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:20.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:30.978 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:40.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:43:51.014 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:01.031 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:11.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:21.070 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:31.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:41.100 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:44:51.111 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:45:01.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:45:11.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:45:33.644 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:45:43.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:45:53.678 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:03.691 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:13.710 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:23.728 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:33.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:43.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:46:53.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:03.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:13.805 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:23.823 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:33.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:43.851 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:47:53.869 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:48:03.886 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:48:13.904 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:48:23.923 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T20:48:33.941 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:48:43.954 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:48:53.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:49:03.991 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:49:14.008 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:49:24.022 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:49:34.040 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:49:44.054 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:49:54.071 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:50:04.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:50:14.098 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:50:24.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:50:34.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T20:50:44.147 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:50:54.165 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:04.184 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:14.199 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:24.213 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:34.222 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:44.239 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:51:54.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:04.278 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:14.288 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:24.301 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:34.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:44.335 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:52:54.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:04.371 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:14.389 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:24.408 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:34.427 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:44.445 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:53:54.464 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:04.480 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:14.495 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:24.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:34.526 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:44.542 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:54:54.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:04.577 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:14.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:24.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:34.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:44.642 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:55:54.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:04.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:14.689 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:24.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:34.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:44.745 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:56:54.761 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:04.778 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:14.796 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:24.812 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:34.828 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:44.846 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:57:54.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:04.876 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:14.894 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:24.925 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:34.936 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:44.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:58:54.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:04.991 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:15.002 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:25.021 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:35.033 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:45.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T20:59:55.068 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:05.087 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:15.104 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:25.122 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:35.140 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:45.151 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:00:55.167 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:05.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:15.195 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:25.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:35.230 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:45.248 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:01:55.261 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:02:05.278 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:02:15.288 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:02:25.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:02:35.371 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:02:45.390 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:02:55.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:05.430 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:15.446 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:25.465 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:35.483 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:45.504 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:03:55.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:05.532 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:15.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:25.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:35.581 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:45.593 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:04:55.606 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:05.625 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:15.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:25.665 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:35.681 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:45.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:05:55.712 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:05.730 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:15.749 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:25.761 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:35.779 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:45.800 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:06:55.817 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:07:05.831 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:07:15.849 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:07:25.869 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:07:35.885 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:07:45.904 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:07:55.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:05.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:15.939 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:25.958 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:35.969 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:45.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:08:56.006 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:06.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:16.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:26.058 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:36.076 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:46.092 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:09:56.112 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:10:06.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:10:16.148 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:10:26.167 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:01.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:11.173 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:21.186 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:31.201 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:41.225 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:11:51.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:01.255 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:11.270 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:21.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:31.302 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:41.320 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:12:51.337 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:01.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:11.368 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:21.388 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:31.406 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:41.425 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:13:51.443 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:14:01.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:14:11.470 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:14:21.489 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:14:31.506 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:14:41.526 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:14:51.544 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:01.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:11.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:21.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:31.616 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:41.634 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:15:51.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:01.680 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:11.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:21.716 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:31.735 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:41.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:16:51.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:01.778 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:11.797 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:21.811 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:31.829 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:41.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:17:51.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:18:01.883 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:18:11.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:18:21.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:18:31.926 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T21:18:41.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:18:51.957 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:01.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:11.994 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:22.012 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:32.030 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:42.049 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:19:52.067 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:02.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:12.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:22.113 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:32.173 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:42.185 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:20:52.204 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:02.219 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:12.237 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:22.256 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:32.274 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:42.282 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:21:52.303 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:06.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:16.596 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:26.613 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:36.632 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:46.651 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:22:56.669 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:06.688 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:16.706 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:26.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:36.742 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:46.757 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:23:56.776 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:06.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:16.816 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:26.836 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:36.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:46.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:24:56.883 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:06.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:16.922 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:26.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:36.946 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:46.965 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:25:56.984 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:26:07.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:26:17.021 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:26:27.039 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:26:37.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:26:47.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:26:57.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:07.104 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:17.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:27.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:37.150 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:47.168 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:27:57.188 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:07.204 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:17.223 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:27.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:37.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:47.276 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:28:57.291 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:07.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:17.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:27.343 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:37.358 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:47.368 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:29:57.386 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:07.403 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:17.422 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:27.432 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:37.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:47.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:30:57.483 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:31:07.493 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:31:17.510 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-11-16T21:31:27.523 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:31:37.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:31:47.560 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:31:57.575 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:07.592 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:17.611 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:27.625 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:37.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:47.659 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:32:57.677 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:07.695 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:17.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:27.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:37.742 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:47.758 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:33:57.770 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:07.786 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:17.802 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:27.820 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:37.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:47.855 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:34:57.867 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:07.886 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:17.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:27.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:37.935 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:47.946 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:35:57.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:07.982 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:17.999 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:28.014 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:38.028 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:48.046 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:36:58.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:08.073 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:18.097 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:28.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:38.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:48.148 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:37:58.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:38:08.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:38:18.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:38:28.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:38:38.234 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:38:48.252 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:38:58.271 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:39:08.285 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:39:18.302 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:39:32.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:39:42.522 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:39:52.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:02.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:12.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:22.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:32.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:42.618 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:40:52.637 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:02.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:12.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:22.689 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:32.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:42.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:41:52.744 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:02.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:12.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:22.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:32.809 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:42.826 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:42:52.844 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:43:02.863 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:43:12.878 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:43:22.897 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:43:32.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:43:42.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:43:52.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:02.962 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:12.981 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:22.999 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:33.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:43.034 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:44:53.050 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:03.068 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:13.087 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:23.104 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:33.118 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:43.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:45:53.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:03.173 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:13.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:23.207 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:33.221 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:43.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:46:53.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:03.252 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:13.267 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:23.279 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:33.292 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:43.307 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:47:53.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:03.344 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:13.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:23.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:33.391 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:43.406 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:48:53.424 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:03.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:13.460 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:23.469 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:33.488 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:43.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:49:53.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:03.543 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:13.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:23.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:33.592 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:43.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:50:53.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:03.646 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:13.657 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:23.676 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:33.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:43.699 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:51:53.714 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:03.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:13.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:23.763 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:33.781 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:43.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:52:53.811 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:03.829 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:13.848 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:23.868 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:33.887 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:43.906 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:53:53.924 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:03.936 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:13.953 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:23.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:33.983 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:44.001 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:54:54.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:04.034 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:14.054 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:24.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:34.087 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:44.106 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:55:54.117 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:04.135 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:14.154 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:24.172 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:34.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:44.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:56:54.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-11-16T21:57:04.238 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:57:14.252 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:57:24.270 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:57:34.288 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:57:44.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:57:54.324 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:04.341 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:14.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:24.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:34.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:44.408 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:58:54.426 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T21:59:04.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:59:14.459 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:59:24.478 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:59:34.490 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:59:44.509 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T21:59:54.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T22:00:04.546 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T22:00:14.563 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-11-16T22:00:24.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:00:34.587 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:00:44.606 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:00:54.618 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:04.636 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:14.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:24.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:34.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:44.704 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:01:54.722 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:04.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:14.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:24.775 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:34.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:44.812 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:02:54.830 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:04.850 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:14.868 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:24.883 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:34.900 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:44.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:03:54.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:04.944 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:14.957 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:24.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:34.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:45.009 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:04:55.029 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:05.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:15.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:25.078 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:35.097 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:45.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:05:55.132 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:05.150 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:15.170 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:25.178 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:35.198 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:45.212 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:06:55.223 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:05.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:15.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:25.268 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:35.287 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:45.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:07:55.321 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:05.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:15.352 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:25.363 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:35.378 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:45.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:08:55.415 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:05.434 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:15.454 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:25.469 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:35.488 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:45.506 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:09:55.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:10:05.542 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-11-16T22:10:15.554 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 4, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024'} 2020-11-16T22:10:15.580 DEBUG:teuthology.lock.ops:locked ubuntu@smithi138.front.sepia.ceph.com, ubuntu@smithi173.front.sepia.ceph.com, ubuntu@smithi194.front.sepia.ceph.com, ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:10:15.687 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f434d7ae840>, signals=[15]) 2020-11-16T22:10:15.892 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.1 2020-11-16T22:10:15.924 INFO:teuthology.provision.fog.smithi194:Scheduling deploy of centos 8.1 2020-11-16T22:10:15.945 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 8.1 2020-11-16T22:10:15.961 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 8.1 2020-11-16T22:10:16.245 INFO:teuthology.orchestra.console:Power off smithi173 2020-11-16T22:10:16.246 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:16.264 INFO:teuthology.orchestra.console:Power off smithi138 2020-11-16T22:10:16.265 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:16.274 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:16.284 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:16.299 INFO:teuthology.orchestra.console:Power off smithi122 2020-11-16T22:10:16.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:16.314 INFO:teuthology.orchestra.console:Power off smithi194 2020-11-16T22:10:16.314 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:10:16.322 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:16.335 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:20.335 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:20.343 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:20.351 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:20.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.450 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.460 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.471 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.479 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.661 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2020-11-16T22:10:24.671 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2020-11-16T22:10:24.691 INFO:teuthology.orchestra.console:Power off for smithi194 completed 2020-11-16T22:10:24.762 INFO:teuthology.orchestra.console:Power on smithi138 2020-11-16T22:10:24.763 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:24.772 INFO:teuthology.orchestra.console:Power on smithi173 2020-11-16T22:10:24.773 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:24.783 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.792 INFO:teuthology.orchestra.console:Power on smithi194 2020-11-16T22:10:24.793 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:24.801 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:24.813 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:28.585 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:28.796 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2020-11-16T22:10:28.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:28.823 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:28.834 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:28.897 INFO:teuthology.orchestra.console:Power on smithi122 2020-11-16T22:10:28.897 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:10:28.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:32.930 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:32.940 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:32.949 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:32.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:33.151 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2020-11-16T22:10:33.201 INFO:teuthology.orchestra.console:Power on for smithi194 completed 2020-11-16T22:10:33.240 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2020-11-16T22:10:33.252 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2020-11-16T22:10:33.302 INFO:teuthology.provision.fog.smithi194:Waiting for deploy to finish 2020-11-16T22:10:33.342 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2020-11-16T22:10:37.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:10:37.269 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2020-11-16T22:10:37.371 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2020-11-16T22:13:04.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:04.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:04.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:08.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:28.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:28.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:28.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:32.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:37.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:37.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:41.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:41.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:47.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:50.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:50.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:50.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:56.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:59.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:59.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:13:59.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:08.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:08.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:33.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:40.766 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:14:40.768 INFO:teuthology.orchestra.run.smithi194:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:14:42.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:14:42.753 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:14:42.754 INFO:teuthology.orchestra.run.smithi138:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:00.707 INFO:teuthology.provision.fog.smithi194:Node is ready 2020-11-16T22:15:00.707 INFO:teuthology.orchestra.run.smithi194:> hostname 2020-11-16T22:15:00.726 INFO:teuthology.orchestra.run.smithi194.stdout:smithi194.front.sepia.ceph.com 2020-11-16T22:15:00.726 INFO:teuthology.orchestra.run.smithi194:> grep smithi194.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:00.782 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.15.194 smithi194.front.sepia.ceph.com smithi194 2020-11-16T22:15:00.783 INFO:teuthology.orchestra.run.smithi194:> sudo hostname smithi194 2020-11-16T22:15:00.910 INFO:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/smithi194.front.sepia.ceph.com/smithi194/g' /etc/hosts 2020-11-16T22:15:00.940 INFO:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/172.21.15.194/172.21.15.194/g' /etc/hosts 2020-11-16T22:15:01.007 INFO:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/smithi194.front.sepia.ceph.com/smithi194/g' /etc/hostname 2020-11-16T22:15:01.075 INFO:teuthology.orchestra.run.smithi194:> sudo hostname smithi194 2020-11-16T22:15:01.141 INFO:teuthology.provision.fog.smithi194:Deploy complete! 2020-11-16T22:15:01.142 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:02.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:03.021 INFO:teuthology.provision.fog.smithi138:Node is ready 2020-11-16T22:15:03.022 INFO:teuthology.orchestra.run.smithi138:> hostname 2020-11-16T22:15:03.040 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2020-11-16T22:15:03.040 INFO:teuthology.orchestra.run.smithi138:> grep smithi138.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:03.096 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2020-11-16T22:15:03.096 INFO:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2020-11-16T22:15:03.105 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:15:03.105 INFO:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:03.263 INFO:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hosts 2020-11-16T22:15:03.293 INFO:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/172.21.15.138/172.21.15.138/g' /etc/hosts 2020-11-16T22:15:03.362 INFO:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hostname 2020-11-16T22:15:03.408 INFO:teuthology.provision.fog.smithi173:Node is ready 2020-11-16T22:15:03.409 INFO:teuthology.orchestra.run.smithi173:> hostname 2020-11-16T22:15:03.422 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2020-11-16T22:15:03.422 INFO:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:03.430 INFO:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2020-11-16T22:15:03.476 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2020-11-16T22:15:03.477 INFO:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2020-11-16T22:15:03.497 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2020-11-16T22:15:03.498 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:03.582 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2020-11-16T22:15:03.610 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2020-11-16T22:15:03.677 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2020-11-16T22:15:03.743 INFO:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2020-11-16T22:15:03.809 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2020-11-16T22:15:03.809 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:05.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:06.230 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:15:06.230 INFO:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:15:06.546 INFO:teuthology.provision.fog.smithi122:Node is ready 2020-11-16T22:15:06.547 INFO:teuthology.orchestra.run.smithi122:> hostname 2020-11-16T22:15:06.562 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com 2020-11-16T22:15:06.562 INFO:teuthology.orchestra.run.smithi122:> grep smithi122.front.sepia.ceph.com /etc/hosts 2020-11-16T22:15:06.618 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2020-11-16T22:15:06.618 INFO:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2020-11-16T22:15:06.730 INFO:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hosts 2020-11-16T22:15:06.759 INFO:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.122/172.21.15.122/g' /etc/hosts 2020-11-16T22:15:06.827 INFO:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hostname 2020-11-16T22:15:06.895 INFO:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2020-11-16T22:15:06.962 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2020-11-16T22:15:06.963 DEBUG:teuthology.parallel:result is None 2020-11-16T22:15:07.158 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2020-11-16T22:15:07.159 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:07.177 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2020-11-16T22:15:07.177 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:07.191 INFO:teuthology.lock.ops:Checking smithi194.front.sepia.ceph.com 2020-11-16T22:15:07.191 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:07.203 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2020-11-16T22:15:07.204 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:15:07.217 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:07.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:07.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:07.287 INFO:teuthology.orchestra.run.smithi138:> true 2020-11-16T22:15:07.448 INFO:teuthology.orchestra.run.smithi138:> uname -m 2020-11-16T22:15:07.500 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2020-11-16T22:15:07.501 INFO:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2020-11-16T22:15:07.554 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Linux" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8 (Core)" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:07.556 INFO:teuthology.orchestra.run.smithi138.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:07.557 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:07.557 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:07.557 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:15:07.558 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2020-11-16T22:15:07.572 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:07.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:07.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:07.640 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-16T22:15:07.803 INFO:teuthology.orchestra.run.smithi173:> uname -m 2020-11-16T22:15:07.857 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2020-11-16T22:15:07.857 INFO:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2020-11-16T22:15:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Linux" 2020-11-16T22:15:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8 (Core)" 2020-11-16T22:15:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2020-11-16T22:15:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:07.911 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:07.912 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:15:07.913 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2020-11-16T22:15:07.930 INFO:teuthology.lock.ops:Updating [smithi194.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:07.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:07.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:07.995 INFO:teuthology.orchestra.run.smithi194:> true 2020-11-16T22:15:08.160 INFO:teuthology.orchestra.run.smithi194:> uname -m 2020-11-16T22:15:08.213 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2020-11-16T22:15:08.213 INFO:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2020-11-16T22:15:08.265 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Linux" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8 (Core)" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:08.266 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:08.267 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:08.268 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:08.268 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:15:08.268 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2020-11-16T22:15:08.285 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:15:08.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:15:08.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:08.351 INFO:teuthology.orchestra.run.smithi122:> true 2020-11-16T22:15:08.512 INFO:teuthology.orchestra.run.smithi122:> uname -m 2020-11-16T22:15:08.566 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2020-11-16T22:15:08.566 INFO:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2020-11-16T22:15:08.620 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Linux" 2020-11-16T22:15:08.620 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8 (Core)" 2020-11-16T22:15:08.620 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:15:08.623 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2020-11-16T22:15:08.638 INFO:teuthology.task.internal.lock_machines:4 smithi machines locked this try, 4/4 locked so far 2020-11-16T22:15:08.701 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi122.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqWOvZqtmsS/gx5H+CiHACrR6ZuO+Vm36ArOBrqTKtow87DYWXruyM/dHsRVy7/bfHyte2QXUhouebGs3wj3rQiG6NCML8dERf/K0MX0HnQ1a1XWDvOgtp2eA+msJafhD0ucB8jwSESxX8qbiN8IK8Hlcu6tpNb01GubNFxMyhOYgUyJxwQPC3jZJ0VCu+OXje9TTE5brjUPZ9gewR5oC57FFcraybBM6NIlQ00usoKDO/nlMZDI+rsX6o5YZ5+tOyaVLYEJYrVthKESHMvOdFfpip153NI2jyeHIbHSxg92cz5L3v5bDZEuNglvyKKwRIco5LesoV8idwJHk0fKe1ymHpmyj25Ze7v2XlhWjM1vTsYgda5227BnOON+oN/eahaTqGoljX76ACl0Ja3wEVNWR8c9IB3rSPIty83SOl6O9IKWVQEn3cGwimoe6DpfXMp6FYmoFFovITuL3h/Stq8aFYtyTo2wwMfLkIY7Mq+Yf+I1I3SXQ+WbTyCejDSLM= smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrzrkjUHYhtPv1T+itDNnKqc54x5/iplG6NjcfGMMJatSRWozQUx5lYwRGndR+OtuOkd2gUTyrpuv56GMAuE/Ubhfo9IXaSBKTy9G2WGFVkF6driH9+s4tldEd6zunrU2Ki6LQMfYzsaLHbXYfqfaLLmrHdSGazHqeQZwQC8Eqq2qYyO6ObG601RyRwIG+qG2oM/bVBxofJ8B0c+lLuBb2500cJB0DTWoB9vIyF2qH1y4+UdQdgIF2WjWdzIv6+oZtPJfA79QP6sY7pPgi42lH5PYU3bNz4BfLZtWnAnk/ACgCVAe+bo5UgrwdMEjBymAcWrx67REJs8IhYbbRFvmExEsJ8Kni0md6sKl73srqRQLv7CsYKtJeswO8X8gePA8ue7wMICNfcvOlkiD0wCMBIKt4K/F3AGIZRYGgJU/vrs5Z0qthPVBnJqasyPNh6vuEKeYo2Sr+P5fNhNzMOt4eJFQfFWdVaJ5Xoq4rU6z3kE8ZpSyeOFLWfFbxRSFlKC0= smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzAAIpAkgVglbuXrxrMaE7OqMYdaw9gudwLBy71h0P505j0+FXcncvcPPoeQ974acr5wVqZtJgLg5wNgN1hSIDEcpoJU07zD0Q1BymiN5oYzeLof3b/Z9sbUr49VzvfSl1RZkga0SG4LlCvuW6jLYqDLYgsfCXpiifbv8zKCU9xPzfcFyqaTdCN3hsb+WZZ81nNrx4UJQeBW/E4nm+PCcvsnnk6rfNyWvnhtP8CR04TC6M08yDDju2sD97+mR/8mZrwxKekEF5LJemVKfczZ81UUgo3lIHredN0KxM63mKGqNgurl64Xcc6DjU15kVm9glBwAsN4YbS5gsISm5FA7cHWQD4bk3XCsD9X/9/aNWvnpJcC6BkuYQtXN3M7RbmtikTXs66c5BNDVmfHJ0+Q3DfeJayTw5xKdJRcGgdotybX6KFsD02N3Hc7kGBsYRkwN06QoYCm2Tvi6LyBfgPNHVSKJ9fU4d4MqHxnJPFv4O5KsXpoW6oCqP/BT4JR1uTBM= smithi194.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZrIGBqIb1Mtn+cTIQwjPZlxvsAnU+gLH6N2C10IFfPCZT9m/VhTk0Kj8d+Y4og8JGi8BobQE3utjznZBdDjHV2nTyWUiG48isOmJ+vvEv3muKyeDLYInu6l3g7C044N+IdAU1VTTTMK3PMI2FfIJXjsCZTatfp1nUhZ/yYSdJUxzIWS1c37X94M28xoY8iWUzJrsFKoh3aiJ0rY+EczIx6wsuqx9uCZrq3P0kYpeaMETjvgKICnKR4qYVnj8lrTIpsTOcXxxot3v463miQtl4GRfGtABhw8nrKgA0EWg1tuPp2kqaE8XoCs0DgDeZ/HOk8JNZF2xQJ8Y3G/Itp2Zw/x2HlNFtnkyHr9TxeC1ZL30t/gaCUaVLPCaYZpeDO2/dw2nWzIOrqoIg74dHq8RPLJzaJjfODlzk1uVPPByhRng8RKr/i2GJlU7Nuiubwl3Mgz7ep7wN+knMLHNi1vPNWrqzpLrUs2UaXr1ocHO8RX+0SFX2bxJQMbkPalW0m90= 2020-11-16T22:15:08.701 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T22:15:08.761 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-16T22:15:08.768 INFO:teuthology.task.internal:Saving configuration 2020-11-16T22:15:08.782 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-16T22:15:08.790 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-16T22:15:08.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:15.575932', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZabxZ7ISLxpHzIg5hYbnDKNJKHgUY891WU2AdBjBawIB56UIxEqMhvFzI9SBK+fzJ56HgCcsQxjbi95f9K/CE=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi138.front.sepia.ceph.com'} 2020-11-16T22:15:08.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:15.575952', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0NxLABItpU476SgDrA1cZbo5/aqiS/BqvrkjplIea32gX7hr66u7cmQTFWv+gt78XVhFw+hbF80fIkrJPeJQ4=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi173.front.sepia.ceph.com'} 2020-11-16T22:15:08.838 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:15.575967', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6TWCpu1URPOX+79EylIDx/0KbIgvyz94W5fW0vuYgcG6g94vE5EnM3PO1XHWRkcGU8Q6s/cY+cwMjwkqt/b+U=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi194.front.sepia.ceph.com'} 2020-11-16T22:15:08.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 22:10:15.575980', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ8b7EbFsXRNAYbT2vwR4tyFTPjffI1jsuQdDn4uNhdL+Oqd146YDsx7pf34GdHMAxmG1Aip2PrcK6wkV7Kc1hw=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi122.front.sepia.ceph.com'} 2020-11-16T22:15:08.854 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-16T22:15:08.861 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'] 2020-11-16T22:15:08.861 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['osd.0'] 2020-11-16T22:15:08.861 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['osd.1'] 2020-11-16T22:15:08.861 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['osd.2'] 2020-11-16T22:15:08.861 INFO:teuthology.run_tasks:Running task console_log... 2020-11-16T22:15:09.006 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4351317e18>, signals=[15]) 2020-11-16T22:15:09.007 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-16T22:15:09.016 INFO:teuthology.task.internal:Opening connections... 2020-11-16T22:15:09.017 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2020-11-16T22:15:09.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:09.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:15:09.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:09.162 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2020-11-16T22:15:09.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:09.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:15:09.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:15:09.304 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-16T22:15:09.313 INFO:teuthology.orchestra.run.smithi138:> uname -m 2020-11-16T22:15:09.328 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2020-11-16T22:15:09.328 INFO:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2020-11-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Linux" 2020-11-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8 (Core)" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:15:09.385 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2020-11-16T22:15:09.401 INFO:teuthology.orchestra.run.smithi173:> uname -m 2020-11-16T22:15:09.417 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2020-11-16T22:15:09.417 INFO:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2020-11-16T22:15:09.470 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Linux" 2020-11-16T22:15:09.470 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8 (Core)" 2020-11-16T22:15:09.470 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:15:09.473 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2020-11-16T22:15:09.488 INFO:teuthology.orchestra.run.smithi194:> uname -m 2020-11-16T22:15:09.502 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2020-11-16T22:15:09.503 INFO:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2020-11-16T22:15:09.555 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Linux" 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8 (Core)" 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8" 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi194.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:09.558 INFO:teuthology.orchestra.run.smithi194.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:09.558 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:09.558 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:09.558 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:15:09.558 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2020-11-16T22:15:09.576 INFO:teuthology.orchestra.run.smithi122:> uname -m 2020-11-16T22:15:09.590 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2020-11-16T22:15:09.590 INFO:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2020-11-16T22:15:09.643 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Linux" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8 (Core)" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2020-11-16T22:15:09.644 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.centos.org/" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-16T22:15:09.645 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-16T22:15:09.646 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-16T22:15:09.646 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:15:09.646 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2020-11-16T22:15:09.661 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-16T22:15:09.697 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-16T22:15:09.708 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-16T22:15:09.708 INFO:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:15:09.711 INFO:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:15:09.713 INFO:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:15:09.715 INFO:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:15:09.727 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-16T22:15:09.738 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-16T22:15:09.738 INFO:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:15:09.763 INFO:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:15:09.766 INFO:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:15:09.768 INFO:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:15:09.799 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-16T22:15:09.878 INFO:teuthology.run_tasks:Running task kernel... 2020-11-16T22:15:09.894 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}} 2020-11-16T22:15:09.894 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-16T22:15:09.895 INFO:teuthology.orchestra.run.smithi138:> uname -r 2020-11-16T22:15:09.909 INFO:teuthology.orchestra.run.smithi138.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:09.910 INFO:teuthology.task.kernel:Running kernel on smithi138: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:09.910 INFO:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2020-11-16T22:15:12.368 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - AppStream 39 MB/s | 7.0 MB 00:00 2020-11-16T22:15:13.549 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - Base 26 MB/s | 2.2 MB 00:00 2020-11-16T22:15:13.974 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-11-16T22:15:14.407 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.3 MB 00:00 2020-11-16T22:15:16.172 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 4.4 MB/s | 23 kB 00:00 2020-11-16T22:15:17.082 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-16T22:15:17.082 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-16T22:15:17.256 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2020-11-16T22:15:17.257 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2020-11-16T22:15:17.257 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2020-11-16T22:15:17.346 INFO:teuthology.orchestra.run.smithi138:> echo no | sudo yum reinstall kernel || true 2020-11-16T22:15:17.750 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:01 ago on Mon 16 Nov 2020 10:15:16 PM UTC. 2020-11-16T22:15:17.988 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:17.989 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:18.034 INFO:teuthology.orchestra.run.smithi138.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:18.080 INFO:teuthology.orchestra.run.smithi138:> sudo yum reinstall -y kernel || true 2020-11-16T22:15:18.481 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Nov 2020 10:15:16 PM UTC. 2020-11-16T22:15:18.729 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:18.730 INFO:teuthology.orchestra.run.smithi138.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:18.764 INFO:teuthology.orchestra.run.smithi138.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:18.811 INFO:teuthology.orchestra.run.smithi138:> rpm -q kernel --last | head -n 1 2020-11-16T22:15:18.867 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-16T22:15:18.869 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:18.869 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-16T22:15:18.870 INFO:teuthology.orchestra.run.smithi173:> uname -r 2020-11-16T22:15:18.885 INFO:teuthology.orchestra.run.smithi173.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:18.886 INFO:teuthology.task.kernel:Running kernel on smithi173: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:18.886 INFO:teuthology.orchestra.run.smithi173:> sudo yum install -y kernel 2020-11-16T22:15:21.380 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - AppStream 42 MB/s | 7.0 MB 00:00 2020-11-16T22:15:22.556 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-16T22:15:22.984 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - Extras 89 kB/s | 5.5 kB 00:00 2020-11-16T22:15:23.412 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.3 MB 00:00 2020-11-16T22:15:25.150 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 4.2 MB/s | 23 kB 00:00 2020-11-16T22:15:26.124 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-16T22:15:26.124 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-16T22:15:26.291 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2020-11-16T22:15:26.292 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2020-11-16T22:15:26.293 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2020-11-16T22:15:26.402 INFO:teuthology.orchestra.run.smithi173:> echo no | sudo yum reinstall kernel || true 2020-11-16T22:15:26.808 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:01 ago on Mon 16 Nov 2020 10:15:25 PM UTC. 2020-11-16T22:15:27.056 INFO:teuthology.orchestra.run.smithi173.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:27.056 INFO:teuthology.orchestra.run.smithi173.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:27.105 INFO:teuthology.orchestra.run.smithi173.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:27.150 INFO:teuthology.orchestra.run.smithi173:> sudo yum reinstall -y kernel || true 2020-11-16T22:15:27.552 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Nov 2020 10:15:25 PM UTC. 2020-11-16T22:15:27.797 INFO:teuthology.orchestra.run.smithi173.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:27.797 INFO:teuthology.orchestra.run.smithi173.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:27.833 INFO:teuthology.orchestra.run.smithi173.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:27.902 INFO:teuthology.orchestra.run.smithi173:> rpm -q kernel --last | head -n 1 2020-11-16T22:15:27.968 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-16T22:15:27.969 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:27.969 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-16T22:15:27.970 INFO:teuthology.orchestra.run.smithi194:> uname -r 2020-11-16T22:15:27.986 INFO:teuthology.orchestra.run.smithi194.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:27.986 INFO:teuthology.task.kernel:Running kernel on smithi194: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:27.986 INFO:teuthology.orchestra.run.smithi194:> sudo yum install -y kernel 2020-11-16T22:15:30.520 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - AppStream 42 MB/s | 7.0 MB 00:00 2020-11-16T22:15:31.732 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - Base 23 MB/s | 2.2 MB 00:00 2020-11-16T22:15:32.164 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-11-16T22:15:32.639 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 19 MB/s | 8.3 MB 00:00 2020-11-16T22:15:34.389 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 4.0 MB/s | 23 kB 00:00 2020-11-16T22:15:35.340 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-16T22:15:35.341 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-16T22:15:35.504 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2020-11-16T22:15:35.505 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2020-11-16T22:15:35.505 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2020-11-16T22:15:35.628 INFO:teuthology.orchestra.run.smithi194:> echo no | sudo yum reinstall kernel || true 2020-11-16T22:15:36.034 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Nov 2020 10:15:34 PM UTC. 2020-11-16T22:15:36.283 INFO:teuthology.orchestra.run.smithi194.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:36.283 INFO:teuthology.orchestra.run.smithi194.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:36.338 INFO:teuthology.orchestra.run.smithi194.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:36.386 INFO:teuthology.orchestra.run.smithi194:> sudo yum reinstall -y kernel || true 2020-11-16T22:15:36.788 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Nov 2020 10:15:34 PM UTC. 2020-11-16T22:15:37.040 INFO:teuthology.orchestra.run.smithi194.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:37.041 INFO:teuthology.orchestra.run.smithi194.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:37.070 INFO:teuthology.orchestra.run.smithi194.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:37.113 INFO:teuthology.orchestra.run.smithi194:> rpm -q kernel --last | head -n 1 2020-11-16T22:15:37.175 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-16T22:15:37.176 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:37.176 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-16T22:15:37.177 INFO:teuthology.orchestra.run.smithi122:> uname -r 2020-11-16T22:15:37.193 INFO:teuthology.orchestra.run.smithi122.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:37.193 INFO:teuthology.task.kernel:Running kernel on smithi122: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:37.194 INFO:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2020-11-16T22:15:39.806 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - AppStream 40 MB/s | 7.0 MB 00:00 2020-11-16T22:15:41.031 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Base 26 MB/s | 2.2 MB 00:00 2020-11-16T22:15:41.462 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Extras 81 kB/s | 5.5 kB 00:00 2020-11-16T22:15:41.869 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 23 MB/s | 8.3 MB 00:00 2020-11-16T22:15:43.628 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 4.8 MB/s | 23 kB 00:00 2020-11-16T22:15:44.602 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-16T22:15:44.602 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-16T22:15:44.764 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2020-11-16T22:15:44.765 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2020-11-16T22:15:44.766 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2020-11-16T22:15:44.900 INFO:teuthology.orchestra.run.smithi122:> echo no | sudo yum reinstall kernel || true 2020-11-16T22:15:45.301 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Nov 2020 10:15:43 PM UTC. 2020-11-16T22:15:45.546 INFO:teuthology.orchestra.run.smithi122.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:45.547 INFO:teuthology.orchestra.run.smithi122.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:45.604 INFO:teuthology.orchestra.run.smithi122.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:45.647 INFO:teuthology.orchestra.run.smithi122:> sudo yum reinstall -y kernel || true 2020-11-16T22:15:46.049 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:03 ago on Mon 16 Nov 2020 10:15:43 PM UTC. 2020-11-16T22:15:46.284 INFO:teuthology.orchestra.run.smithi122.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-16T22:15:46.284 INFO:teuthology.orchestra.run.smithi122.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-16T22:15:46.344 INFO:teuthology.orchestra.run.smithi122.stderr:Error: No packages marked for reinstall. 2020-11-16T22:15:46.406 INFO:teuthology.orchestra.run.smithi122:> rpm -q kernel --last | head -n 1 2020-11-16T22:15:46.468 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-16T22:15:46.470 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-16T22:15:46.470 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-16T22:15:46.470 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-16T22:15:46.471 INFO:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T22:15:46.688 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2020-11-16T22:15:46.701 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2020-11-16T22:15:46.701 INFO:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T22:15:46.743 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2020-11-16T22:15:46.755 INFO:teuthology.task.kernel:Enabling kdb on osd.1... 2020-11-16T22:15:46.755 INFO:teuthology.orchestra.run.smithi194:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T22:15:46.795 INFO:teuthology.orchestra.run.smithi194.stdout:ttyS1 2020-11-16T22:15:46.807 INFO:teuthology.task.kernel:Enabling kdb on osd.2... 2020-11-16T22:15:46.807 INFO:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T22:15:46.840 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2020-11-16T22:15:46.856 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-16T22:15:46.863 INFO:teuthology.task.internal:Creating test directory... 2020-11-16T22:15:46.863 INFO:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:15:46.898 INFO:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:15:46.900 INFO:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:15:46.903 INFO:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:15:46.918 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-16T22:15:46.926 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-16T22:15:46.938 INFO:teuthology.task.internal:Creating archive directory... 2020-11-16T22:15:46.938 INFO:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:15:46.953 INFO:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:15:46.957 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:15:46.959 INFO:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:15:47.009 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-16T22:15:47.015 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-16T22:15:47.016 INFO:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:15:47.018 INFO:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:15:47.019 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:15:47.034 INFO:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:15:47.050 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.065 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.065 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.066 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.071 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.079 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.080 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.086 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:15:47.087 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-16T22:15:47.094 INFO:teuthology.task.internal:Configuring sudo... 2020-11-16T22:15:47.094 INFO:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:15:47.096 INFO:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:15:47.108 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:15:47.130 INFO:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:15:47.159 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-16T22:15:47.168 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-16T22:15:47.168 INFO:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:15:47.171 INFO:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:15:47.178 INFO:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:15:47.199 INFO:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:15:47.214 INFO:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.265 INFO:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.338 INFO:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.371 INFO:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.422 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:15:47.423 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:47.474 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:47.487 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:15:47.487 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:15:47.487 INFO:teuthology.orchestra.run.smithi138.stderr:152 bytes copied, 2.9681e-05 s, 5.1 MB/s 2020-11-16T22:15:47.488 INFO:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.581 INFO:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.653 INFO:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.705 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:15:47.705 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:47.758 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:47.771 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:15:47.771 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:15:47.772 INFO:teuthology.orchestra.run.smithi173.stderr:152 bytes copied, 2.9237e-05 s, 5.2 MB/s 2020-11-16T22:15:47.772 INFO:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.808 INFO:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:47.866 INFO:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.939 INFO:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:47.992 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:15:47.993 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:48.046 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:48.060 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:15:48.060 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:15:48.060 INFO:teuthology.orchestra.run.smithi194.stderr:152 bytes copied, 3.2066e-05 s, 4.7 MB/s 2020-11-16T22:15:48.061 INFO:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:48.094 INFO:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:15:48.156 INFO:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:48.229 INFO:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:15:48.282 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:15:48.283 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:15:48.349 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:15:48.349 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:15:48.349 INFO:teuthology.orchestra.run.smithi122.stderr:152 bytes copied, 2.9163e-05 s, 5.2 MB/s 2020-11-16T22:15:48.350 INFO:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2020-11-16T22:15:48.392 INFO:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2020-11-16T22:15:48.394 INFO:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2020-11-16T22:15:48.397 INFO:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2020-11-16T22:15:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-16T22:15:48.472 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-16T22:15:48.477 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-16T22:15:48.481 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-16T22:15:48.892 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-16T22:15:48.901 INFO:teuthology.task.internal:Starting timer... 2020-11-16T22:15:48.901 INFO:teuthology.run_tasks:Running task pcp... 2020-11-16T22:15:48.918 INFO:teuthology.run_tasks:Running task selinux... 2020-11-16T22:15:48.931 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-16T22:15:48.932 INFO:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2020-11-16T22:15:48.989 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: [ OK ] 2020-11-16T22:15:48.991 INFO:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2020-11-16T22:15:49.067 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: [ OK ] 2020-11-16T22:15:49.068 INFO:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2020-11-16T22:15:49.140 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: [ OK ] 2020-11-16T22:15:49.142 INFO:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2020-11-16T22:15:49.214 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: [ OK ] 2020-11-16T22:15:49.216 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-16T22:15:49.216 INFO:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2020-11-16T22:15:49.242 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2020-11-16T22:15:49.243 INFO:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2020-11-16T22:15:49.269 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2020-11-16T22:15:49.270 INFO:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2020-11-16T22:15:49.288 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2020-11-16T22:15:49.288 INFO:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2020-11-16T22:15:49.317 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2020-11-16T22:15:49.317 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2020-11-16T22:15:49.318 INFO:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-16T22:15:49.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:15:49.349 INFO:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-16T22:15:49.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:15:49.380 INFO:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-16T22:15:49.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:15:49.411 INFO:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-16T22:15:49.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:15:49.439 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-16T22:15:49.440 INFO:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2020-11-16T22:15:49.472 INFO:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2020-11-16T22:15:49.502 INFO:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2020-11-16T22:15:49.533 INFO:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2020-11-16T22:15:49.572 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-16T22:15:49.582 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-11-16T22:15:57.850 INFO:teuthology.repo_utils:Fetching master from origin 2020-11-16T22:15:57.900 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-16T22:15:57.914 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-16T22:15:57.916 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi173.front.sepia.ceph.com,smithi194.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-16T22:16:14.759 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-16T22:16:19.710 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-16T22:16:26.098 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-16T22:16:26.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:26 +0000 (0:00:00.066) 0:00:00.066 ******* 2020-11-16T22:16:26.132 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:26 +0000 (0:00:00.015) 0:00:00.081 ******* 2020-11-16T22:16:26.148 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:26 +0000 (0:00:00.015) 0:00:00.097 ******* 2020-11-16T22:16:26.163 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:26 +0000 (0:00:00.015) 0:00:00.113 ******* 2020-11-16T22:16:31.684 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi122.front.sepia.ceph.com] 2020-11-16T22:16:31.749 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:31 +0000 (0:00:05.585) 0:00:05.698 ******* 2020-11-16T22:16:44.126 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] 2020-11-16T22:16:44.134 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] 2020-11-16T22:16:44.141 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] 2020-11-16T22:16:44.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:12.494) 0:00:18.193 ******* 2020-11-16T22:16:44.272 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.028) 0:00:18.222 ******* 2020-11-16T22:16:44.300 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.028) 0:00:18.250 ******* 2020-11-16T22:16:44.430 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-16T22:16:44.432 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:16:44.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.162) 0:00:18.413 ******* 2020-11-16T22:16:44.471 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:16:44.472 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:16:44.504 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.040) 0:00:18.453 ******* 2020-11-16T22:16:44.533 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.028) 0:00:18.482 ******* 2020-11-16T22:16:44.540 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:16:44.572 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.039) 0:00:18.522 ******* 2020-11-16T22:16:44.869 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-16T22:16:44.871 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} ok: [smithi138.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T22:16:44.871 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T22:16:44.873 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T22:16:44.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.330) 0:00:18.853 ******* 2020-11-16T22:16:44.931 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.027) 0:00:18.881 ******* 2020-11-16T22:16:44.960 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.028) 0:00:18.909 ******* 2020-11-16T22:16:44.987 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:44 +0000 (0:00:00.027) 0:00:18.937 ******* 2020-11-16T22:16:44.992 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-16T22:16:44.994 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:45.025 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.038) 0:00:18.975 ******* 2020-11-16T22:16:45.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.027) 0:00:19.002 ******* 2020-11-16T22:16:45.080 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.027) 0:00:19.030 ******* 2020-11-16T22:16:45.089 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:45.122 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.041) 0:00:19.072 ******* 2020-11-16T22:16:45.326 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-16T22:16:45.328 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:16:45.331 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:16:45.332 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:16:45.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.240) 0:00:19.312 ******* 2020-11-16T22:16:45.391 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.028) 0:00:19.340 ******* 2020-11-16T22:16:45.420 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.028) 0:00:19.369 ******* 2020-11-16T22:16:45.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:45 +0000 (0:00:00.030) 0:00:19.400 ******* 2020-11-16T22:16:46.161 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-16T22:16:46.163 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565005.4330971-172934682780109/source", "state": "file", "uid": 0} changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565005.3749135-91954904585719/source", "state": "file", "uid": 0} 2020-11-16T22:16:46.164 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565005.4657707-39494708567593/source", "state": "file", "uid": 0} 2020-11-16T22:16:46.166 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605565005.404894-119222920263850/source", "state": "file", "uid": 0} 2020-11-16T22:16:46.215 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.764) 0:00:20.164 ******* 2020-11-16T22:16:46.250 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.035) 0:00:20.200 ******* 2020-11-16T22:16:46.280 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.029) 0:00:20.229 ******* 2020-11-16T22:16:46.308 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.028) 0:00:20.257 ******* 2020-11-16T22:16:46.585 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-16T22:16:46.587 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-16T22:16:46.588 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-16T22:16:46.616 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-16T22:16:46.618 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-11-16T22:16:46.620 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-11-16T22:16:46.621 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEz 2020-11-16T22:16:46.623 INFO:teuthology.task.ansible.out:IKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-16T22:16:46.790 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-16T22:16:46.797 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-16T22:16:46.922 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.614) 0:00:20.871 ******* 2020-11-16T22:16:46.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.029) 0:00:20.901 ******* 2020-11-16T22:16:46.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:46 +0000 (0:00:00.029) 0:00:20.930 ******* 2020-11-16T22:16:47.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:47 +0000 (0:00:00.028) 0:00:20.959 ******* 2020-11-16T22:16:47.558 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi122.front.sepia.ceph.com] 2020-11-16T22:16:47.638 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-16T22:16:47.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:47 +0000 (0:00:00.627) 0:00:21.587 ******* 2020-11-16T22:16:47.647 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] 2020-11-16T22:16:47.655 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] 2020-11-16T22:16:47.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:47 +0000 (0:00:00.105) 0:00:21.693 ******* 2020-11-16T22:16:47.778 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:47 +0000 (0:00:00.033) 0:00:21.727 ******* 2020-11-16T22:16:48.597 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] 2020-11-16T22:16:48.665 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:48 +0000 (0:00:00.887) 0:00:22.614 ******* 2020-11-16T22:16:56.445 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-16T22:16:56.447 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:08.477422", "end": "2020-11-16 22:16:56.420789", "rc": 0, "start": "2020-11-16 22:16:47.943367", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} changed: [smithi122.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:08.505377", "end": "2020-11-16 22:16:56.420703", "rc": 0, "start": "2020-11-16 22:16:47.915326", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:56.448 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:08.511106", "end": "2020-11-16 22:16:56.423674", "rc": 0, "start": "2020-11-16 22:16:47.912568", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:56.479 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:07.814) 0:00:30.429 ******* 2020-11-16T22:16:56.513 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.033) 0:00:30.462 ******* 2020-11-16T22:16:56.543 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.030) 0:00:30.492 ******* 2020-11-16T22:16:56.684 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T22:16:56.686 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011478", "end": "2020-11-16 22:16:56.680492", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:16:56.669014", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:56.718 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.174) 0:00:30.667 ******* 2020-11-16T22:16:56.725 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011836", "end": "2020-11-16 22:16:56.713380", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:16:56.701544", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:56.757 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.039) 0:00:30.707 ******* 2020-11-16T22:16:56.763 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:16:56.765 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.006393", "end": "2020-11-16 22:16:56.740606", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:16:56.734213", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:56.796 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.038) 0:00:30.746 ******* 2020-11-16T22:16:56.825 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.028) 0:00:30.774 ******* 2020-11-16T22:16:56.831 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:16:56.832 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:56.863 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.038) 0:00:30.812 ******* 2020-11-16T22:16:56.891 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.028) 0:00:30.841 ******* 2020-11-16T22:16:56.897 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:16:56.899 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:56.929 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.037) 0:00:30.878 ******* 2020-11-16T22:16:56.956 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.027) 0:00:30.906 ******* 2020-11-16T22:16:56.961 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:16:56.963 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:56.993 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:56 +0000 (0:00:00.036) 0:00:30.942 ******* 2020-11-16T22:16:57.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.027) 0:00:30.970 ******* 2020-11-16T22:16:57.027 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:16:57.028 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.058 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.037) 0:00:31.008 ******* 2020-11-16T22:16:57.088 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.029) 0:00:31.037 ******* 2020-11-16T22:16:57.094 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T22:16:57.095 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.126 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.038) 0:00:31.076 ******* 2020-11-16T22:16:57.154 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.028) 0:00:31.104 ******* 2020-11-16T22:16:57.161 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T22:16:57.162 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.196 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.041) 0:00:31.146 ******* 2020-11-16T22:16:57.203 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:57.204 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.205 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.206 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.206 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b037 2020-11-16T22:16:57.208 INFO:teuthology.task.ansible.out:4849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f821 2020-11-16T22:16:57.209 INFO:teuthology.task.ansible.out:5edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.239 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.042) 0:00:31.189 ******* 2020-11-16T22:16:57.246 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.247 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T22:16:57.247 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:57.248 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.248 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was Fals 2020-11-16T22:16:57.250 INFO:teuthology.task.ansible.out:e"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1 2020-11-16T22:16:57.252 INFO:teuthology.task.ansible.out:EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d 2020-11-16T22:16:57.253 INFO:teuthology.task.ansible.out:44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.281 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.042) 0:00:31.231 ******* 2020-11-16T22:16:57.287 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.288 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T22:16:57.288 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:57.289 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.289 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.290 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyu 2020-11-16T22:16:57.292 INFO:teuthology.task.ansible.out:riw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.041) 0:00:31.272 ******* 2020-11-16T22:16:57.329 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.330 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.330 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.332 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.333 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi138.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:08.470564", "end": "2020-11-16 22:16:57.294367", "rc": 0, "start": "2020-11-16 22:16:48.823803", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:16:57.335 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af 2020-11-16T22:16:57.336 INFO:teuthology.task.ansible.out:05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.043) 0:00:31.316 ******* 2020-11-16T22:16:57.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.029) 0:00:31.345 ******* 2020-11-16T22:16:57.402 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.404 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T22:16:57.404 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.404 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.404 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.406 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.406 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.408 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.408 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS 2020-11-16T22:16:57.409 INFO:teuthology.task.ansible.out:4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.441 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.045) 0:00:31.390 ******* 2020-11-16T22:16:57.447 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.448 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T22:16:57.449 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.449 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.449 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.451 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.451 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.451 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': ' 2020-11-16T22:16:57.453 INFO:teuthology.task.ansible.out:gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.043) 0:00:31.434 ******* 2020-11-16T22:16:57.491 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.492 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.494 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.494 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.494 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.495 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS 2020-11-16T22:16:57.496 INFO:teuthology.task.ansible.out:8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95 2020-11-16T22:16:57.498 INFO:teuthology.task.ansible.out:d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87a 2020-11-16T22:16:57.499 INFO:teuthology.task.ansible.out:aa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.509 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.510 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.521 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.522 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.523 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA 2020-11-16T22:16:57.524 INFO:teuthology.task.ansible.out:AABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.534 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.536 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQ 2020-11-16T22:16:57.538 INFO:teuthology.task.ansible.out:xreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.547 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.547 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.547 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.548 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa 2020-11-16T22:16:57.549 INFO:teuthology.task.ansible.out:7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d0 2020-11-16T22:16:57.551 INFO:teuthology.task.ansible.out:59fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.560 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.562 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ 2020-11-16T22:16:57.563 INFO:teuthology.task.ansible.out: ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577 2020-11-16T22:16:57.565 INFO:teuthology.task.ansible.out:a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.574 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.576 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T22:16:57.576 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013082", "end": "2020-11-16 22:16:57.564879", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:16:57.551797", " 2020-11-16T22:16:57.577 INFO:teuthology.task.ansible.out:stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item 2020-11-16T22:16:57.579 INFO:teuthology.task.ansible.out:": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.125) 0:00:31.559 ******* 2020-11-16T22:16:57.617 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.617 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.619 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.620 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575c 2020-11-16T22:16:57.622 INFO:teuthology.task.ansible.out:cd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@offi 2020-11-16T22:16:57.623 INFO:teuthology.task.ansible.out:ce w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b 2020-11-16T22:16:57.625 INFO:teuthology.task.ansible.out:83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822a 2020-11-16T22:16:57.626 INFO:teuthology.task.ansible.out:f8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae18905745 2020-11-16T22:16:57.628 INFO:teuthology.task.ansible.out:77a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason" 2020-11-16T22:16:57.629 INFO:teuthology.task.ansible.out:: "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.638 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.639 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.640 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zy 2020-11-16T22:16:57.641 INFO:teuthology.task.ansible.out:an@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": fals 2020-11-16T22:16:57.643 INFO:teuthology.task.ansible.out:e, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "ite 2020-11-16T22:16:57.644 INFO:teuthology.task.ansible.out:m", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.675 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.065) 0:00:31.624 ******* 2020-11-16T22:16:57.682 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.683 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.684 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.685 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.686 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.686 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "da 2020-11-16T22:16:57.688 INFO:teuthology.task.ansible.out:ng", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.688 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f3 2020-11-16T22:16:57.689 INFO:teuthology.task.ansible.out:9ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reaso 2020-11-16T22:16:57.691 INFO:teuthology.task.ansible.out:n": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.701 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.703 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.705 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.707 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.707 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'owasserm', 2020-11-16T22:16:57.708 INFO:teuthology.task.ansible.out: 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec 2020-11-16T22:16:57.710 INFO:teuthology.task.ansible.out:5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca456 2020-11-16T22:16:57.711 INFO:teuthology.task.ansible.out:3399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.743 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.068) 0:00:31.692 ******* 2020-11-16T22:16:57.751 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.751 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.753 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.753 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.753 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macp 2020-11-16T22:16:57.755 INFO:teuthology.task.ansible.out:rasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.755 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-de 2020-11-16T22:16:57.757 INFO:teuthology.task.ansible.out:sktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51 2020-11-16T22:16:57.758 INFO:teuthology.task.ansible.out:fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, " 2020-11-16T22:16:57.760 INFO:teuthology.task.ansible.out:skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.771 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.772 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.773 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.773 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "o 2020-11-16T22:16:57.774 INFO:teuthology.task.ansible.out:vpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.775 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item= 2020-11-16T22:16:57.776 INFO:teuthology.task.ansible.out:{'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd 2020-11-16T22:16:57.777 INFO:teuthology.task.ansible.out:4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.778 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39R 2020-11-16T22:16:57.779 INFO:teuthology.task.ansible.out:sFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.781 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.813 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.069) 0:00:31.762 ******* 2020-11-16T22:16:57.820 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.821 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.822 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.823 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.825 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.825 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.826 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.826 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed" 2020-11-16T22:16:57.827 INFO:teuthology.task.ansible.out:: false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.829 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.829 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.830 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.831 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dori 2020-11-16T22:16:57.832 INFO:teuthology.task.ansible.out:nda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.848 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.850 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.850 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.891 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.078) 0:00:31.841 ******* 2020-11-16T22:16:57.922 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.031) 0:00:31.872 ******* 2020-11-16T22:16:57.930 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.931 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.932 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.932 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.934 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.934 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.934 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31Gb 2020-11-16T22:16:57.936 INFO:teuthology.task.ansible.out:Do9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.936 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.936 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd48742 2020-11-16T22:16:57.938 INFO:teuthology.task.ansible.out:80b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd0 2020-11-16T22:16:57.939 INFO:teuthology.task.ansible.out:9606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.940 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d5 2020-11-16T22:16:57.941 INFO:teuthology.task.ansible.out:07442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8 2020-11-16T22:16:57.943 INFO:teuthology.task.ansible.out:d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.943 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.943 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55f 2020-11-16T22:16:57.945 INFO:teuthology.task.ansible.out:cc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi 2020-11-16T22:16:57.946 INFO:teuthology.task.ansible.out:194.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85 2020-11-16T22:16:57.947 INFO:teuthology.task.ansible.out:\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:57.979 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:57 +0000 (0:00:00.056) 0:00:31.928 ******* 2020-11-16T22:16:58.009 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.030) 0:00:31.958 ******* 2020-11-16T22:16:58.015 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.017 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.018 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.018 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.020 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.020 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.021 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c0 2020-11-16T22:16:58.022 INFO:teuthology.task.ansible.out:92f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.022 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedor 2020-11-16T22:16:58.024 INFO:teuthology.task.ansible.out:a NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a8 2020-11-16T22:16:58.025 INFO:teuthology.task.ansible.out:5b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.025 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.026 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d 2020-11-16T22:16:58.027 INFO:teuthology.task.ansible.out:303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.028 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item 2020-11-16T22:16:58.029 INFO:teuthology.task.ansible.out:", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.029 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94p 2020-11-16T22:16:58.031 INFO:teuthology.task.ansible.out:rpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.067 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.057) 0:00:32.016 ******* 2020-11-16T22:16:58.098 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.031) 0:00:32.048 ******* 2020-11-16T22:16:58.105 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.107 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.108 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.109 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.109 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.109 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX 2020-11-16T22:16:58.111 INFO:teuthology.task.ansible.out:7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.111 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.112 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.113 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.113 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": 2020-11-16T22:16:58.118 INFO:teuthology.task.ansible.out: "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.119 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj 2020-11-16T22:16:58.120 INFO:teuthology.task.ansible.out:4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJ 2020-11-16T22:16:58.122 INFO:teuthology.task.ansible.out:wWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f 2020-11-16T22:16:58.123 INFO:teuthology.task.ansible.out:7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": f 2020-11-16T22:16:58.125 INFO:teuthology.task.ansible.out:alse, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia. 2020-11-16T22:16:58.126 INFO:teuthology.task.ansible.out:ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.157 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.059) 0:00:32.107 ******* 2020-11-16T22:16:58.165 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.166 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.168 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.169 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.169 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.171 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.171 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.173 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.173 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.173 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade2 2020-11-16T22:16:58.175 INFO:teuthology.task.ansible.out:6b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.176 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.178 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.178 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.178 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.193 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.195 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.208 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.209 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-16T22:16:58.209 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.209 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.210 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.210 INFO:teuthology.task.ansible.out:skipping: [smithi138.fron 2020-11-16T22:16:58.211 INFO:teuthology.task.ansible.out:t.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbP 2020-11-16T22:16:58.213 INFO:teuthology.task.ansible.out:FyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet 2020-11-16T22:16:58.214 INFO:teuthology.task.ansible.out:", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.242 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.084) 0:00:32.192 ******* 2020-11-16T22:16:58.272 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.030) 0:00:32.222 ******* 2020-11-16T22:16:58.301 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.028) 0:00:32.251 ******* 2020-11-16T22:16:58.308 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.309 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.311 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.312 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.312 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96 2020-11-16T22:16:58.314 INFO:teuthology.task.ansible.out:a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": 2020-11-16T22:16:58.315 INFO:teuthology.task.ansible.out: "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", 2020-11-16T22:16:58.317 INFO:teuthology.task.ansible.out: "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f84 2020-11-16T22:16:58.318 INFO:teuthology.task.ansible.out:4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikH 2020-11-16T22:16:58.320 INFO:teuthology.task.ansible.out:CWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** 2020-11-16T22:16:58.350 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.048) 0:00:32.300 ******* 2020-11-16T22:16:58.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.029) 0:00:32.330 ******* 2020-11-16T22:16:58.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.028) 0:00:32.358 ******* 2020-11-16T22:16:58.414 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.416 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.418 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.418 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.418 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "v 2020-11-16T22:16:58.420 INFO:teuthology.task.ansible.out:asu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.420 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.421 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mwatts', 2020-11-16T22:16:58.422 INFO:teuthology.task.ansible.out: 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat. 2020-11-16T22:16:58.424 INFO:teuthology.task.ansible.out:com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.424 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfa 2020-11-16T22:16:58.425 INFO:teuthology.task.ansible.out:b5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.456 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.046) 0:00:32.405 ******* 2020-11-16T22:16:58.492 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.036) 0:00:32.441 ******* 2020-11-16T22:16:58.499 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-16T22:16:58.500 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.502 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.503 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.505 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.505 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.505 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli 2020-11-16T22:16:58.507 INFO:teuthology.task.ansible.out:0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.507 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.508 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoe 2020-11-16T22:16:58.509 INFO:teuthology.task.ansible.out:n", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.510 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc1 2020-11-16T22:16:58.511 INFO:teuthology.task.ansible.out:86056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.546 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.053) 0:00:32.495 ******* 2020-11-16T22:16:58.552 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.553 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.554 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.555 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.556 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.557 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.557 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.558 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.558 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional re 2020-11-16T22:16:58.559 INFO:teuthology.task.ansible.out:sult was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.560 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.560 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se 2020-11-16T22:16:58.562 INFO:teuthology.task.ansible.out:0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.593 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.047) 0:00:32.542 ******* 2020-11-16T22:16:58.600 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.601 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.602 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.603 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} TASK [users : Create all adm 2020-11-16T22:16:58.604 INFO:teuthology.task.ansible.out:in users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.605 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.605 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f 2020-11-16T22:16:58.607 INFO:teuthology.task.ansible.out:8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.608 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] *********** 2020-11-16T22:16:58.609 INFO:teuthology.task.ansible.out:************* skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": { 2020-11-16T22:16:58.611 INFO:teuthology.task.ansible.out:"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.049) 0:00:32.592 ******* 2020-11-16T22:16:58.649 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.650 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.651 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.652 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.652 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.652 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.652 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.653 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.653 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] * 2020-11-16T22:16:58.654 INFO:teuthology.task.ansible.out:******************************************* skipping: [smithi122.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.655 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.655 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.655 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.656 INFO:teuthology.task.ansible.out: TASK [users : 2020-11-16T22:16:58.657 INFO:teuthology.task.ansible.out: Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.657 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5 2020-11-16T22:16:58.659 INFO:teuthology.task.ansible.out:ea9994b91b"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.659 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": " 2020-11-16T22:16:58.661 INFO:teuthology.task.ansible.out:Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.691 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.048) 0:00:32.641 ******* 2020-11-16T22:16:58.697 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.699 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.700 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.700 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.701 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [s 2020-11-16T22:16:58.703 INFO:teuthology.task.ansible.out:mithi138.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.703 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jf 2020-11-16T22:16:58.705 INFO:teuthology.task.ansible.out:suselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.707 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:16:58.709 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasa 2020-11-16T22:16:58.711 INFO:teuthology.task.ansible.out:d", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.049) 0:00:32.691 ******* 2020-11-16T22:16:58.771 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.029) 0:00:32.720 ******* 2020-11-16T22:16:58.777 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T22:16:58.779 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.779 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.780 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dcain', 'ov 2020-11-16T22:16:58.782 INFO:teuthology.task.ansible.out:pn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.782 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570c 2020-11-16T22:16:58.784 INFO:teuthology.task.ansible.out:be42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf 2020-11-16T22:16:58.786 INFO:teuthology.task.ansible.out:3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.816 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.045) 0:00:32.766 ******* 2020-11-16T22:16:58.823 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.824 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T22:16:58.824 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.825 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.826 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.827 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.827 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdha 2020-11-16T22:16:58.828 INFO:teuthology.task.ansible.out:nge", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.842 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.056) 0:00:32.822 ******* 2020-11-16T22:16:58.901 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.028) 0:00:32.851 ******* 2020-11-16T22:16:58.930 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.028) 0:00:32.880 ******* 2020-11-16T22:16:58.936 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-16T22:16:58.938 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:58.940 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.971 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:58 +0000 (0:00:00.040) 0:00:32.920 ******* 2020-11-16T22:16:58.978 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.979 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** 2020-11-16T22:16:58.980 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.981 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.981 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.982 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:58.982 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.013 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.041) 0:00:32.962 ******* 2020-11-16T22:16:59.023 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.024 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-16T22:16:59.025 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.025 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.026 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.026 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.027 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.027 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.065 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.052) 0:00:33.015 ******* 2020-11-16T22:16:59.073 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.075 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.076 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.076 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.077 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.077 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (it 2020-11-16T22:16:59.079 INFO:teuthology.task.ansible.out:em=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.080 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.080 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.091 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.093 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi194.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.094 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.126 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.060) 0:00:33.075 ******* 2020-11-16T22:16:59.133 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.135 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.136 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.137 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.137 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.138 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.138 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] 2020-11-16T22:16:59.140 INFO:teuthology.task.ansible.out:=> (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.141 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.141 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.152 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.156 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.157 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi194.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.189 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.063) 0:00:33.139 ******* 2020-11-16T22:16:59.222 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.032) 0:00:33.172 ******* 2020-11-16T22:16:59.229 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.230 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.232 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.232 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.232 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.233 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.234 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.234 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.236 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi194.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.236 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.236 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.237 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.237 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.271 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.048) 0:00:33.220 ******* 2020-11-16T22:16:59.278 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.279 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.315 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.044) 0:00:33.265 ******* 2020-11-16T22:16:59.322 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-16T22:16:59.323 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.041) 0:00:33.307 ******* 2020-11-16T22:16:59.395 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:59.397 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.406 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.408 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.418 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.420 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.429 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.431 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.441 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.443 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.452 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.454 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.463 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.465 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.475 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.476 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.486 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.488 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.498 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.499 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} TASK [users : Remove revoked users] ******************************************** skipping: [smithi138.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.533 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.175) 0:00:33.482 ******* 2020-11-16T22:16:59.542 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.543 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.544 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.544 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.545 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.545 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.545 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.546 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.557 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.558 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi122.front.sepia.ceph.com] => { "msg": "Host smithi122.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-16T22:16:59.591 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.058) 0:00:33.541 ******* 2020-11-16T22:16:59.598 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:16:59.598 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.600 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.600 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.601 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.602 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.045) 0:00:33.586 ******* 2020-11-16T22:16:59.645 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:16:59.647 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-16T22:16:59.678 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.041) 0:00:33.628 ******* 2020-11-16T22:16:59.695 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:16:59.697 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-16T22:16:59.727 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.049) 0:00:33.677 ******* 2020-11-16T22:16:59.734 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T22:16:59.735 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.040) 0:00:33.717 ******* 2020-11-16T22:16:59.773 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T22:16:59.776 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => { "msg": "Host smithi173.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-16T22:16:59.776 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:16:59.777 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-16T22:16:59.808 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.040) 0:00:33.758 ******* 2020-11-16T22:16:59.841 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.032) 0:00:33.790 ******* 2020-11-16T22:16:59.847 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:16:59.848 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-16T22:16:59.879 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.038) 0:00:33.829 ******* 2020-11-16T22:16:59.885 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T22:16:59.886 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => { "msg": "Host smithi194.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-16T22:16:59.917 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.037) 0:00:33.867 ******* 2020-11-16T22:16:59.923 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T22:16:59.924 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:16:59.955 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.038) 0:00:33.905 ******* 2020-11-16T22:16:59.961 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:16:59.963 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-16T22:16:59.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:16:59 +0000 (0:00:00.038) 0:00:33.944 ******* 2020-11-16T22:17:00.021 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T22:17:00.022 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.058) 0:00:34.002 ******* 2020-11-16T22:17:00.100 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:17:00.102 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-16T22:17:00.135 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.082) 0:00:34.084 ******* 2020-11-16T22:17:00.162 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T22:17:00.163 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => { "msg": "Host smithi138.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-16T22:17:00.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.057) 0:00:34.142 ******* 2020-11-16T22:17:00.230 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:17:00.231 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-16T22:17:00.260 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.067) 0:00:34.210 ******* 2020-11-16T22:17:00.287 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T22:17:00.289 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.318 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.057) 0:00:34.267 ******* 2020-11-16T22:17:00.323 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:17:00.325 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.340309", "end": "2020-11-16 22:17:00.290532", "rc": 0, "start": "2020-11-16 22:16:59.950223", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:17:00.356 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.038) 0:00:34.306 ******* 2020-11-16T22:17:00.384 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T22:17:00.385 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.058) 0:00:34.365 ******* 2020-11-16T22:17:00.442 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T22:17:00.444 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.473 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.057) 0:00:34.423 ******* 2020-11-16T22:17:00.480 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:17:00.482 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.316655", "end": "2020-11-16 22:17:00.458771", "rc": 0, "start": "2020-11-16 22:17:00.142116", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:17:00.513 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.040) 0:00:34.463 ******* 2020-11-16T22:17:00.519 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T22:17:00.520 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.036) 0:00:34.500 ******* 2020-11-16T22:17:00.556 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T22:17:00.557 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.588 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.037) 0:00:34.538 ******* 2020-11-16T22:17:00.595 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-16T22:17:00.596 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.038) 0:00:34.576 ******* 2020-11-16T22:17:00.634 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:17:00.636 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.359860", "end": "2020-11-16 22:17:00.596724", "rc": 0, "start": "2020-11-16 22:17:00.236864", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.666 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.039) 0:00:34.616 ******* 2020-11-16T22:17:00.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.035) 0:00:34.651 ******* 2020-11-16T22:17:00.707 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:17:00.709 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.740 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.038) 0:00:34.689 ******* 2020-11-16T22:17:00.769 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.028) 0:00:34.718 ******* 2020-11-16T22:17:00.774 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T22:17:00.776 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.808 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.039) 0:00:34.757 ******* 2020-11-16T22:17:00.837 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.029) 0:00:34.786 ******* 2020-11-16T22:17:00.843 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-16T22:17:00.844 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.845 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi122.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:17:00.845 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.334784", "end": "2020-11-16 22:17:00.835848", "rc": 0, "start": "2020-11-16 22:17:00.501064", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:17:00.877 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.040) 0:00:34.827 ******* 2020-11-16T22:17:00.906 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.028) 0:00:34.855 ******* 2020-11-16T22:17:00.935 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.029) 0:00:34.884 ******* 2020-11-16T22:17:00.942 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T22:17:00.943 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:00.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:00 +0000 (0:00:00.046) 0:00:34.930 ******* 2020-11-16T22:17:01.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.028) 0:00:34.959 ******* 2020-11-16T22:17:01.038 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.028) 0:00:34.988 ******* 2020-11-16T22:17:01.045 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:17:01.046 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.078 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.039) 0:00:35.027 ******* 2020-11-16T22:17:01.108 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.030) 0:00:35.058 ******* 2020-11-16T22:17:01.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.088 ******* 2020-11-16T22:17:01.144 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T22:17:01.146 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.176 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.037) 0:00:35.126 ******* 2020-11-16T22:17:01.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.040) 0:00:35.166 ******* 2020-11-16T22:17:01.245 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.027) 0:00:35.194 ******* 2020-11-16T22:17:01.250 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T22:17:01.251 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.038) 0:00:35.233 ******* 2020-11-16T22:17:01.313 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.262 ******* 2020-11-16T22:17:01.343 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.292 ******* 2020-11-16T22:17:01.348 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.349 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-16T22:17:01.349 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:17:01.350 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.385 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.042) 0:00:35.335 ******* 2020-11-16T22:17:01.414 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.028) 0:00:35.363 ******* 2020-11-16T22:17:01.444 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.393 ******* 2020-11-16T22:17:01.451 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-16T22:17:01.452 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.452 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.039) 0:00:35.433 ******* 2020-11-16T22:17:01.511 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.027) 0:00:35.461 ******* 2020-11-16T22:17:01.539 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.027) 0:00:35.489 ******* 2020-11-16T22:17:01.546 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:17:01.548 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi138.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:01.548 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:01.579 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.040) 0:00:35.529 ******* 2020-11-16T22:17:01.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.559 ******* 2020-11-16T22:17:01.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.027) 0:00:35.587 ******* 2020-11-16T22:17:01.643 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T22:17:01.644 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set have_entitlements] ****************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.038) 0:00:35.625 ******* 2020-11-16T22:17:01.704 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.027) 0:00:35.653 ******* 2020-11-16T22:17:01.733 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.682 ******* 2020-11-16T22:17:01.739 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T22:17:01.740 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.772 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.039) 0:00:35.722 ******* 2020-11-16T22:17:01.801 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.028) 0:00:35.750 ******* 2020-11-16T22:17:01.829 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.028) 0:00:35.778 ******* 2020-11-16T22:17:01.834 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.835 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T22:17:01.835 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:01.835 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:01.867 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.038) 0:00:35.817 ******* 2020-11-16T22:17:01.898 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.030) 0:00:35.847 ******* 2020-11-16T22:17:01.929 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.031) 0:00:35.878 ******* 2020-11-16T22:17:01.934 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-16T22:17:01.936 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:17:01.936 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:01.968 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.039) 0:00:35.917 ******* 2020-11-16T22:17:01.998 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:01 +0000 (0:00:00.029) 0:00:35.947 ******* 2020-11-16T22:17:02.028 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:35.977 ******* 2020-11-16T22:17:02.034 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-16T22:17:02.035 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:02.070 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.042) 0:00:36.020 ******* 2020-11-16T22:17:02.099 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.048 ******* 2020-11-16T22:17:02.128 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.077 ******* 2020-11-16T22:17:02.134 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T22:17:02.135 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.168 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.039) 0:00:36.117 ******* 2020-11-16T22:17:02.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.147 ******* 2020-11-16T22:17:02.226 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.175 ******* 2020-11-16T22:17:02.231 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:17:02.232 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.265 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.039) 0:00:36.214 ******* 2020-11-16T22:17:02.294 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.243 ******* 2020-11-16T22:17:02.322 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.272 ******* 2020-11-16T22:17:02.329 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-16T22:17:02.331 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.362 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.040) 0:00:36.312 ******* 2020-11-16T22:17:02.391 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.341 ******* 2020-11-16T22:17:02.421 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.370 ******* 2020-11-16T22:17:02.428 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-16T22:17:02.429 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.461 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.039) 0:00:36.410 ******* 2020-11-16T22:17:02.491 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.440 ******* 2020-11-16T22:17:02.520 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.469 ******* 2020-11-16T22:17:02.526 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-16T22:17:02.528 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:02.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.042) 0:00:36.511 ******* 2020-11-16T22:17:02.592 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.541 ******* 2020-11-16T22:17:02.620 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.570 ******* 2020-11-16T22:17:02.625 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T22:17:02.627 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.628 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.039) 0:00:36.609 ******* 2020-11-16T22:17:02.689 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.638 ******* 2020-11-16T22:17:02.717 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.667 ******* 2020-11-16T22:17:02.724 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:17:02.726 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.761 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.043) 0:00:36.710 ******* 2020-11-16T22:17:02.790 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.739 ******* 2020-11-16T22:17:02.819 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.029) 0:00:36.769 ******* 2020-11-16T22:17:02.825 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T22:17:02.827 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.858 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.038) 0:00:36.808 ******* 2020-11-16T22:17:02.887 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.837 ******* 2020-11-16T22:17:02.893 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:02.895 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.926 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.038) 0:00:36.876 ******* 2020-11-16T22:17:02.954 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.028) 0:00:36.904 ******* 2020-11-16T22:17:02.960 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T22:17:02.960 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:02.962 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** ok: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:03.003 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:02 +0000 (0:00:00.048) 0:00:36.952 ******* 2020-11-16T22:17:03.031 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.028) 0:00:36.981 ******* 2020-11-16T22:17:03.061 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.029) 0:00:37.010 ******* 2020-11-16T22:17:03.068 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T22:17:03.070 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.041) 0:00:37.052 ******* 2020-11-16T22:17:03.134 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.031) 0:00:37.083 ******* 2020-11-16T22:17:03.140 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T22:17:03.141 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:03.173 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.123 ******* 2020-11-16T22:17:03.205 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.031) 0:00:37.154 ******* 2020-11-16T22:17:03.211 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T22:17:03.212 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.244 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.193 ******* 2020-11-16T22:17:03.271 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.027) 0:00:37.221 ******* 2020-11-16T22:17:03.277 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T22:17:03.279 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.310 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.038) 0:00:37.259 ******* 2020-11-16T22:17:03.339 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.029) 0:00:37.289 ******* 2020-11-16T22:17:03.346 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T22:17:03.348 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.379 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.328 ******* 2020-11-16T22:17:03.385 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:17:03.387 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.418 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.367 ******* 2020-11-16T22:17:03.432 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T22:17:03.434 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.045) 0:00:37.413 ******* 2020-11-16T22:17:03.469 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T22:17:03.470 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.471 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-16T22:17:03.604 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.140) 0:00:37.554 ******* 2020-11-16T22:17:03.611 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T22:17:03.613 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Increase the yum timeout.] ************************************** ok: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:03.646 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.041) 0:00:37.595 ******* 2020-11-16T22:17:03.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.029) 0:00:37.625 ******* 2020-11-16T22:17:03.681 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-16T22:17:03.683 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.665 ******* 2020-11-16T22:17:03.723 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:17:03.725 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.755 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.705 ******* 2020-11-16T22:17:03.761 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-16T22:17:03.762 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.794 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.038) 0:00:37.744 ******* 2020-11-16T22:17:03.801 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T22:17:03.803 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.833 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.038) 0:00:37.782 ******* 2020-11-16T22:17:03.839 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-16T22:17:03.841 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.039) 0:00:37.822 ******* 2020-11-16T22:17:03.880 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-16T22:17:03.881 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:03.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.040) 0:00:37.863 ******* 2020-11-16T22:17:03.920 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T22:17:03.922 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-16T22:17:03.956 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.042) 0:00:37.905 ******* 2020-11-16T22:17:03.986 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:03 +0000 (0:00:00.030) 0:00:37.936 ******* 2020-11-16T22:17:03.993 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-16T22:17:03.995 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.026 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.040) 0:00:37.976 ******* 2020-11-16T22:17:04.032 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-16T22:17:04.034 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:04.065 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.038) 0:00:38.015 ******* 2020-11-16T22:17:04.072 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-16T22:17:04.074 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.105 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.039) 0:00:38.055 ******* 2020-11-16T22:17:04.112 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-16T22:17:04.114 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-16T22:17:04.147 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.041) 0:00:38.097 ******* 2020-11-16T22:17:04.154 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T22:17:04.156 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.187 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.040) 0:00:38.137 ******* 2020-11-16T22:17:04.194 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-16T22:17:04.195 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.228 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.040) 0:00:38.177 ******* 2020-11-16T22:17:04.234 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-16T22:17:04.235 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:17:04.267 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.038) 0:00:38.216 ******* 2020-11-16T22:17:04.295 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-16T22:17:04.297 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.326 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.059) 0:00:38.275 ******* 2020-11-16T22:17:04.362 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-16T22:17:04.364 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.399 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.073) 0:00:38.348 ******* 2020-11-16T22:17:04.416 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:17:04.417 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-16T22:17:04.438 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-16T22:17:04.439 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:04.470 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.071) 0:00:38.420 ******* 2020-11-16T22:17:04.550 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:17:04.552 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-16T22:17:04.585 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.114) 0:00:38.534 ******* 2020-11-16T22:17:04.613 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-16T22:17:04.615 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:04.645 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.060) 0:00:38.595 ******* 2020-11-16T22:17:04.861 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:17:04.862 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-16T22:17:04.899 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.253) 0:00:38.848 ******* 2020-11-16T22:17:04.926 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:17:04.928 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:04.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:04 +0000 (0:00:00.058) 0:00:38.907 ******* 2020-11-16T22:17:04.965 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:17:04.965 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-16T22:17:04.987 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:04.989 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.018 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.060) 0:00:38.968 ******* 2020-11-16T22:17:05.045 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:17:05.047 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi138.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-16T22:17:05.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.058) 0:00:39.027 ******* 2020-11-16T22:17:05.105 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:17:05.107 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.059) 0:00:39.086 ******* 2020-11-16T22:17:05.144 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:17:05.145 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-16T22:17:05.156 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.186 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.049) 0:00:39.136 ******* 2020-11-16T22:17:05.213 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:17:05.214 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.056) 0:00:39.193 ******* 2020-11-16T22:17:05.269 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:17:05.271 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.300 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.057) 0:00:39.250 ******* 2020-11-16T22:17:05.327 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:17:05.329 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.358 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.057) 0:00:39.308 ******* 2020-11-16T22:17:05.375 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:17:05.377 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-16T22:17:05.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.050) 0:00:39.359 ******* 2020-11-16T22:17:05.415 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:17:05.416 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.041) 0:00:39.401 ******* 2020-11-16T22:17:05.479 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:17:05.481 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-16T22:17:05.514 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.062) 0:00:39.463 ******* 2020-11-16T22:17:05.552 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.583 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.069) 0:00:39.533 ******* 2020-11-16T22:17:05.589 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:17:05.591 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-16T22:17:05.624 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.040) 0:00:39.573 ******* 2020-11-16T22:17:05.629 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:05.630 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.662 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.038) 0:00:39.611 ******* 2020-11-16T22:17:05.690 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:17:05.691 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.720 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.058) 0:00:39.670 ******* 2020-11-16T22:17:05.747 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:17:05.748 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.778 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.058) 0:00:39.728 ******* 2020-11-16T22:17:05.806 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:05.807 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.837 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.058) 0:00:39.786 ******* 2020-11-16T22:17:05.864 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:17:05.866 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.896 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.058) 0:00:39.845 ******* 2020-11-16T22:17:05.903 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:17:05.905 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-16T22:17:05.915 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:05.946 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.050) 0:00:39.895 ******* 2020-11-16T22:17:05.973 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:17:05.975 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.004 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:05 +0000 (0:00:00.057) 0:00:39.953 ******* 2020-11-16T22:17:06.030 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:17:06.032 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.060 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.056) 0:00:40.010 ******* 2020-11-16T22:17:06.098 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi194.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-16T22:17:06.319 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-16T22:17:06.354 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.293) 0:00:40.303 ******* 2020-11-16T22:17:06.381 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:17:06.382 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.412 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.058) 0:00:40.362 ******* 2020-11-16T22:17:06.439 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:06.441 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.470 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.057) 0:00:40.420 ******* 2020-11-16T22:17:06.497 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:17:06.499 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.532 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.061) 0:00:40.481 ******* 2020-11-16T22:17:06.539 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:17:06.540 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.870755", "end": "2020-11-16 22:17:06.500335", "rc": 0, "start": "2020-11-16 22:17:05.629580", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.258594", "end": "2020-11-16 22:17:06.500349", "rc": 0, "start": "2020-11-16 22:17:06.241755", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:06.541 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T22:17:06.541 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-16T22:17:06.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.042) 0:00:40.524 ******* 2020-11-16T22:17:06.604 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.029) 0:00:40.553 ******* 2020-11-16T22:17:06.633 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.029) 0:00:40.583 ******* 2020-11-16T22:17:06.640 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:17:06.641 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.674 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.040) 0:00:40.624 ******* 2020-11-16T22:17:06.681 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:17:06.683 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.714 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.039) 0:00:40.663 ******* 2020-11-16T22:17:06.721 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:06.722 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.039) 0:00:40.703 ******* 2020-11-16T22:17:06.759 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:06.761 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.792 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.741 ******* 2020-11-16T22:17:06.797 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:17:06.798 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.830 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.779 ******* 2020-11-16T22:17:06.835 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:17:06.837 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.868 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.818 ******* 2020-11-16T22:17:06.874 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:17:06.875 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.907 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.856 ******* 2020-11-16T22:17:06.913 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:17:06.915 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.946 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.895 ******* 2020-11-16T22:17:06.952 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:17:06.953 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:06.984 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:06 +0000 (0:00:00.038) 0:00:40.934 ******* 2020-11-16T22:17:06.991 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:17:06.992 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.025 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.040) 0:00:40.974 ******* 2020-11-16T22:17:07.031 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:17:07.032 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.063 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.038) 0:00:41.013 ******* 2020-11-16T22:17:07.070 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T22:17:07.071 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.105 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.041) 0:00:41.055 ******* 2020-11-16T22:17:07.132 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:17:07.134 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.164 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.058) 0:00:41.113 ******* 2020-11-16T22:17:07.190 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:17:07.191 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.221 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.057) 0:00:41.171 ******* 2020-11-16T22:17:07.248 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:17:07.250 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.279 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.057) 0:00:41.229 ******* 2020-11-16T22:17:07.508 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:17:07.510 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.258290", "end": "2020-11-16 22:17:07.500356", "rc": 0, "start": "2020-11-16 22:17:07.242066", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.038301", "end": "2020-11-16 22:17:07.500346", "rc": 0, "start": "2020-11-16 22:17:07.462045", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:07.540 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.260) 0:00:41.489 ******* 2020-11-16T22:17:07.572 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.031) 0:00:41.521 ******* 2020-11-16T22:17:07.642 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:17:07.644 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-16T22:17:07.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.103) 0:00:41.625 ******* 2020-11-16T22:17:07.706 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.030) 0:00:41.655 ******* 2020-11-16T22:17:07.712 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:17:07.714 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.745 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.038) 0:00:41.694 ******* 2020-11-16T22:17:07.751 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.037) 0:00:41.731 ******* 2020-11-16T22:17:07.787 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:17:07.789 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.819 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.037) 0:00:41.769 ******* 2020-11-16T22:17:07.827 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.857 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.037) 0:00:41.807 ******* 2020-11-16T22:17:07.863 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:17:07.865 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.895 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.037) 0:00:41.844 ******* 2020-11-16T22:17:07.901 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.933 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.038) 0:00:41.883 ******* 2020-11-16T22:17:07.940 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:17:07.941 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:07.971 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:07 +0000 (0:00:00.037) 0:00:41.921 ******* 2020-11-16T22:17:07.977 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:41.960 ******* 2020-11-16T22:17:08.415 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:17:08.417 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:17:08.450 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.439) 0:00:42.399 ******* 2020-11-16T22:17:08.458 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:17:08.459 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} TASK [common : Add krb5 config file] ******************************************* ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:17:08.492 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.042) 0:00:42.441 ******* 2020-11-16T22:17:08.532 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.039) 0:00:42.481 ******* 2020-11-16T22:17:08.539 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:17:08.540 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-16T22:17:08.572 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.040) 0:00:42.522 ******* 2020-11-16T22:17:08.580 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:17:08.582 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.619 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.046) 0:00:42.568 ******* 2020-11-16T22:17:08.626 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.657 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:42.607 ******* 2020-11-16T22:17:08.664 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:17:08.665 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.696 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:42.646 ******* 2020-11-16T22:17:08.703 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.735 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:42.684 ******* 2020-11-16T22:17:08.741 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:17:08.742 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.773 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:42.723 ******* 2020-11-16T22:17:08.780 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.038) 0:00:42.762 ******* 2020-11-16T22:17:08.818 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:17:08.819 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.850 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.037) 0:00:42.799 ******* 2020-11-16T22:17:08.856 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:08.890 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:08 +0000 (0:00:00.040) 0:00:42.839 ******* 2020-11-16T22:17:09.290 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:17:09.292 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:17:09.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:09 +0000 (0:00:00.433) 0:00:43.273 ******* 2020-11-16T22:17:09.340 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:17:09.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:09 +0000 (0:00:00.049) 0:00:43.322 ******* 2020-11-16T22:17:11.710 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:17:11.712 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-16T22:17:11.742 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:11 +0000 (0:00:02.369) 0:00:45.691 ******* 2020-11-16T22:17:11.884 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-16T22:17:11.916 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:11 +0000 (0:00:00.174) 0:00:45.866 ******* 2020-11-16T22:17:12.272 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:12.274 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:17:12.418 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:17:12.555 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:17:12.557 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-16T22:17:12.587 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:12 +0000 (0:00:00.670) 0:00:46.537 ******* 2020-11-16T22:17:12.614 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-16T22:17:12.647 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:12 +0000 (0:00:00.059) 0:00:46.596 ******* 2020-11-16T22:17:12.736 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:12.737 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:17:12.902 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:17:13.100 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:17:13.164 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:17:13.185 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:17:13.219 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:13 +0000 (0:00:00.572) 0:00:47.169 ******* 2020-11-16T22:17:13.350 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:17:13.385 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:13 +0000 (0:00:00.165) 0:00:47.334 ******* 2020-11-16T22:17:13.412 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:17:13.413 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:17:13.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:13 +0000 (0:00:00.060) 0:00:47.395 ******* 2020-11-16T22:17:13.566 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:13.567 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:17:13.577 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:17:13.579 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:17:13.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:13 +0000 (0:00:00.164) 0:00:47.559 ******* 2020-11-16T22:17:13.658 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:13.660 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:17:13.903 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:13.905 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:17:14.029 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:14.031 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:17:14.063 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.452) 0:00:48.012 ******* 2020-11-16T22:17:14.079 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:14.081 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:17:14.102 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:17:14.103 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:17:14.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.075) 0:00:48.087 ******* 2020-11-16T22:17:14.259 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:17:14.261 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:17:14.294 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.156) 0:00:48.244 ******* 2020-11-16T22:17:14.333 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:17:14.334 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:14.334 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:17:14.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.072) 0:00:48.316 ******* 2020-11-16T22:17:14.520 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:17:14.752 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:17:14.787 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.420) 0:00:48.737 ******* 2020-11-16T22:17:14.795 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:17:14.817 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:17:14.819 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:14.847 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.060) 0:00:48.797 ******* 2020-11-16T22:17:14.854 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:14.856 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:17:14.877 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:17:14.879 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-16T22:17:14.908 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.060) 0:00:48.858 ******* 2020-11-16T22:17:14.936 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:14.938 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:17:14.971 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:14 +0000 (0:00:00.062) 0:00:48.921 ******* 2020-11-16T22:17:14.999 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.030 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.058) 0:00:48.979 ******* 2020-11-16T22:17:15.057 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:17:15.059 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-16T22:17:15.088 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.058) 0:00:49.038 ******* 2020-11-16T22:17:15.105 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:17:15.106 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-16T22:17:15.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.048) 0:00:49.086 ******* 2020-11-16T22:17:15.175 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:17:15.177 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.186 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:15.188 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} TASK [common : Install nrpe without epel] ************************************** skipping: [smithi138.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.198 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.209 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.240 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.103) 0:00:49.189 ******* 2020-11-16T22:17:15.267 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:15.269 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:17:15.279 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi122.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-16T22:17:15.310 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.070) 0:00:49.260 ******* 2020-11-16T22:17:15.358 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:17:15.360 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.369 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.371 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.380 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.416 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.105) 0:00:49.365 ******* 2020-11-16T22:17:15.608 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:15.610 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:17:15.642 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.226) 0:00:49.591 ******* 2020-11-16T22:17:15.669 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:17:15.671 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.059) 0:00:49.651 ******* 2020-11-16T22:17:15.709 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:17:15.710 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:17:15.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.041) 0:00:49.693 ******* 2020-11-16T22:17:15.750 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:17:15.751 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-16T22:17:15.783 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.039) 0:00:49.732 ******* 2020-11-16T22:17:15.789 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:17:15.791 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:15.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.038) 0:00:49.771 ******* 2020-11-16T22:17:15.859 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:17:15.860 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-16T22:17:15.890 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:15 +0000 (0:00:00.068) 0:00:49.840 ******* 2020-11-16T22:17:15.979 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:17:15.981 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-16T22:17:16.011 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.120) 0:00:49.961 ******* 2020-11-16T22:17:16.048 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:17:16.050 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.059 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.070 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.071 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.082 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:17:16.084 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-16T22:17:16.113 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.102) 0:00:50.063 ******* 2020-11-16T22:17:16.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.029) 0:00:50.092 ******* 2020-11-16T22:17:16.160 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:17:16.162 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-16T22:17:16.193 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.050) 0:00:50.143 ******* 2020-11-16T22:17:16.200 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:17:16.201 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.202 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.233 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.040) 0:00:50.183 ******* 2020-11-16T22:17:16.239 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:17:16.241 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.276 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.042) 0:00:50.225 ******* 2020-11-16T22:17:16.324 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:17:16.325 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-16T22:17:16.356 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.079) 0:00:50.305 ******* 2020-11-16T22:17:16.384 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:17:16.386 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:16.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.058) 0:00:50.364 ******* 2020-11-16T22:17:16.723 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:17:16.725 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-16T22:17:16.760 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.345) 0:00:50.710 ******* 2020-11-16T22:17:16.871 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-16T22:17:16.904 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:16 +0000 (0:00:00.143) 0:00:50.853 ******* 2020-11-16T22:17:17.046 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:17:17.048 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-16T22:17:17.080 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.175) 0:00:51.029 ******* 2020-11-16T22:17:17.108 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:17:17.109 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:17.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.058) 0:00:51.088 ******* 2020-11-16T22:17:17.166 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:17:17.168 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-16T22:17:17.200 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.061) 0:00:51.150 ******* 2020-11-16T22:17:17.227 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:17:17.229 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:17.259 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.059) 0:00:51.209 ******* 2020-11-16T22:17:17.289 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.029) 0:00:51.238 ******* 2020-11-16T22:17:17.305 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:17:17.307 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:17.339 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.050) 0:00:51.288 ******* 2020-11-16T22:17:17.367 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:17:17.369 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:17:17.401 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.061) 0:00:51.350 ******* 2020-11-16T22:17:17.439 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:17:17.441 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:17.470 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.069) 0:00:51.420 ******* 2020-11-16T22:17:17.542 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:17:17.544 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:17.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.103) 0:00:51.523 ******* 2020-11-16T22:17:17.601 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:17:17.603 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-16T22:17:17.634 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.059) 0:00:51.583 ******* 2020-11-16T22:17:17.672 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:17:17.674 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:17.705 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.070) 0:00:51.654 ******* 2020-11-16T22:17:17.742 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:17:17.744 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-16T22:17:17.775 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.069) 0:00:51.724 ******* 2020-11-16T22:17:17.802 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:17:17.804 INFO:teuthology.task.ansible.out:fatal: [smithi138.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-16T22:17:17.834 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.059) 0:00:51.784 ******* 2020-11-16T22:17:17.872 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:17:17.873 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:17.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.074) 0:00:51.858 ******* 2020-11-16T22:17:17.926 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:17:17.927 INFO:teuthology.task.ansible.out:fatal: [smithi122.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-16T22:17:17.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:17 +0000 (0:00:00.048) 0:00:51.907 ******* 2020-11-16T22:17:17.985 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.016 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.058) 0:00:51.966 ******* 2020-11-16T22:17:18.095 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:17:18.097 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:17:18.128 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.111) 0:00:52.078 ******* 2020-11-16T22:17:18.156 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:17:18.158 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.167 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.198 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.070) 0:00:52.148 ******* 2020-11-16T22:17:18.215 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:17:18.217 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:17:18.248 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.049) 0:00:52.197 ******* 2020-11-16T22:17:18.286 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:17:18.288 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.298 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.330 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.081) 0:00:52.279 ******* 2020-11-16T22:17:18.367 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:17:18.369 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-16T22:17:18.402 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.071) 0:00:52.351 ******* 2020-11-16T22:17:18.408 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:17:18.410 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:18.441 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.039) 0:00:52.391 ******* 2020-11-16T22:17:18.479 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:17:18.480 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-16T22:17:18.512 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.070) 0:00:52.461 ******* 2020-11-16T22:17:18.529 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:17:18.531 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:17:18.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.050) 0:00:52.511 ******* 2020-11-16T22:17:18.682 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:17:18.684 INFO:teuthology.task.ansible.out:fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-16T22:17:18.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.153) 0:00:52.665 ******* 2020-11-16T22:17:18.743 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:17:18.745 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.774 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.058) 0:00:52.724 ******* 2020-11-16T22:17:18.791 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:17:18.793 INFO:teuthology.task.ansible.out:fatal: [smithi194.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-16T22:17:18.823 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.048) 0:00:52.773 ******* 2020-11-16T22:17:18.849 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:17:18.851 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:17:18.880 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:18 +0000 (0:00:00.056) 0:00:52.830 ******* 2020-11-16T22:17:19.085 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:17:19.085 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:31 UTC", "ActiveEnterTimestampMonotonic": "15349475", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target system.slice var-run.mount basic.target tmp.mount -.mount network.target systemd-journald.socket time-sync.target local-fs.target nss-lookup.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:31 UTC", "AssertTimestampMonotonic": "15348683", "Before": "shutdown.target xdm.service getty@tty1.service plymouth-quit.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:31 UTC", "ConditionTimestampMonotonic": "15348683", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-16T22:17:19.087 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2490", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:31 UTC", "ExecMainStartTimestampMonotonic": "15349441", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:31 UTC", "InactiveExitTimestampMonotonic": "15349475", "InvocationID": "6b67dab453804db6a863c5b4f7bfac94", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-16T22:17:19.089 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2490", "MemoryAccounting": "yes", "MemoryCurrent": "1200128", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-16T22:17:19.090 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:31 UTC", "StateChangeTimestampMonotonic": "15349475", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:31 UTC", "WatchdogTimestampMonotonic": "15349474", "WatchdogUSec": "0"}} ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:37 UTC", "ActiveEnterTimestampMonotonic": "15469053", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target -.mount systemd-tmpfiles-setup.service local-fs.target time-sync.target basic.target systemd-journald.socket var-run.mount tmp.mount sysinit.target system.slice network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:37 2020-11-16T22:17:19.092 INFO:teuthology.task.ansible.out:UTC", "AssertTimestampMonotonic": "15468079", "Before": "getty@tty1.service plymouth-quit.service xdm.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:37 UTC", "ConditionTimestampMonotonic": "15468078", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2505", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:37 UTC", "ExecMainStartTimestampMonotonic": "15469018", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv 2020-11-16T22:17:19.093 INFO:teuthology.task.ansible.out:[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:37 UTC", "InactiveExitTimestampMonotonic": "15469053", "InvocationID": "c59cb23c634c4c76bde5f6d8d20ab6f8", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425 2020-11-16T22:17:19.094 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2505", "MemoryAccounting": "yes", "MemoryCurrent": "1204224", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:37 UTC", "StateChangeTimestampMonotonic": "1546 2020-11-16T22:17:19.096 INFO:teuthology.task.ansible.out:9053", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:37 UTC", "WatchdogTimestampMonotonic": "15469052", "WatchdogUSec": "0"}} 2020-11-16T22:17:19.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.240) 0:00:53.070 ******* 2020-11-16T22:17:19.151 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.029) 0:00:53.100 ******* 2020-11-16T22:17:19.231 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:17:19.233 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-16T22:17:19.264 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.113) 0:00:53.214 ******* 2020-11-16T22:17:19.313 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:17:19.314 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001742", "end": "2020-11-16 22:17:19.302198", "rc": 0, "start": "2020-11-16 22:17:19.300456", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-16T22:17:19.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.081) 0:00:53.296 ******* 2020-11-16T22:17:19.353 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:17:19.355 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} TASK [common : Get SELinux status] ********************************************* changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001655", "end": "2020-11-16 22:17:19.335367", "rc": 0, "start": "2020-11-16 22:17:19.333712", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-16T22:17:19.387 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.041) 0:00:53.337 ******* 2020-11-16T22:17:19.417 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.029) 0:00:53.366 ******* 2020-11-16T22:17:19.570 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:17:19.571 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:29 UTC", "ActiveEnterTimestampMonotonic": "15665609", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target time-sync.target var-run.mount system.slice network.target -.mount systemd-journald.socket nss-lookup.target local-fs.target tmp.mount systemd-tmpfiles-setup.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:29 UTC", "AssertTimestampMonotonic": "15664790", "Before": "multi-user.target getty@tty1.service xdm.service shutdown.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:29 UTC", "ConditionTimestampMonotonic": "15664788", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-16T22:17:19.572 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2510", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:29 UTC", "ExecMainStartTimestampMonotonic": "15665583", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:29 UTC", "InactiveExitTimestampMonotonic": "15665609", "InvocationID": "71652df7f2a142c8a38e5fa7ff5ad5fa", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-16T22:17:19.574 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2510", "MemoryAccounting": "yes", "MemoryCurrent": "1286144", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-16T22:17:19.575 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:29 UTC", "StateChangeTimestampMonotonic": "15665609", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:29 UTC", "WatchdogTimestampMonotonic": "15665608", "WatchdogUSec": "0"}} 2020-11-16T22:17:19.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.185) 0:00:53.552 ******* 2020-11-16T22:17:19.716 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:35 UTC", "ActiveEnterTimestampMonotonic": "15193538", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount time-sync.target local-fs.target systemd-journald.socket systemd-tmpfiles-setup.service system.slice basic.target -.mount sysinit.target nss-lookup.target tmp.mount network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:35 UTC", "AssertTimestampMonotonic": "15192559", "Before": "xdm.service getty@tty1.service shutdown.target multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:35 UTC", "ConditionTimestampMonotonic": "15192559", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-16T22:17:19.718 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2499", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:35 UTC", "ExecMainStartTimestampMonotonic": "15193491", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:35 UTC", "InactiveExitTimestampMonotonic": "15193538", "InvocationID": "a6fe96488f3141d4b822b53aea3b226f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-16T22:17:19.720 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2499", "MemoryAccounting": "yes", "MemoryCurrent": "1568768", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-16T22:17:19.721 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:35 UTC", "StateChangeTimestampMonotonic": "15193538", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:35 UTC", "WatchdogTimestampMonotonic": "15193532", "WatchdogUSec": "0"}} 2020-11-16T22:17:19.748 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.145) 0:00:53.698 ******* 2020-11-16T22:17:19.796 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:17:19.798 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001719", "end": "2020-11-16 22:17:19.787779", "rc": 0, "start": "2020-11-16 22:17:19.786060", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-16T22:17:19.829 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.080) 0:00:53.778 ******* 2020-11-16T22:17:19.950 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001674", "end": "2020-11-16 22:17:19.938200", "rc": 0, "start": "2020-11-16 22:17:19.936526", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-16T22:17:19.982 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:19 +0000 (0:00:00.153) 0:00:53.931 ******* 2020-11-16T22:17:20.247 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T22:17:20.248 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-16T22:17:20.281 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:20 +0000 (0:00:00.298) 0:00:54.230 ******* 2020-11-16T22:17:20.303 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-16T22:17:20.335 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:20 +0000 (0:00:00.053) 0:00:54.284 ******* 2020-11-16T22:17:20.759 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-16T22:17:20.792 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:20 +0000 (0:00:00.456) 0:00:54.741 ******* 2020-11-16T22:17:20.903 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-16T22:17:20.940 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:20 +0000 (0:00:00.148) 0:00:54.890 ******* 2020-11-16T22:17:21.218 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-16T22:17:21.219 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-16T22:17:21.251 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:21 +0000 (0:00:00.310) 0:00:55.201 ******* 2020-11-16T22:17:21.269 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-16T22:17:21.302 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:21 +0000 (0:00:00.050) 0:00:55.252 ******* 2020-11-16T22:17:21.725 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-16T22:17:21.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:21 +0000 (0:00:00.455) 0:00:55.707 ******* 2020-11-16T22:17:21.859 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-16T22:17:21.892 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:21 +0000 (0:00:00.133) 0:00:55.841 ******* 2020-11-16T22:17:22.668 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-16T22:17:22.670 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-16T22:17:22.701 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:22 +0000 (0:00:00.809) 0:00:56.650 ******* 2020-11-16T22:17:22.904 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-16T22:17:22.937 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:22 +0000 (0:00:00.235) 0:00:56.886 ******* 2020-11-16T22:17:22.943 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-16T22:17:22.976 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:22 +0000 (0:00:00.039) 0:00:56.925 ******* 2020-11-16T22:17:23.024 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-16T22:17:23.057 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:23 +0000 (0:00:00.080) 0:00:57.006 ******* 2020-11-16T22:17:39.086 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T22:17:39.088 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.189065", "end": "2020-11-16 22:17:39.073506", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:17:22.884441", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-16T22:17:39.118 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:16.061) 0:01:13.068 ******* 2020-11-16T22:17:39.406 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.276153", "end": "2020-11-16 22:17:39.393639", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:17:23.117486", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-16T22:17:39.438 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:00.320) 0:01:13.388 ******* 2020-11-16T22:17:39.467 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.299035", "end": "2020-11-16 22:17:39.456310", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:17:23.157275", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-16T22:17:39.501 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:00.062) 0:01:13.450 ******* 2020-11-16T22:17:39.520 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.267686", "end": "2020-11-16 22:17:39.503201", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:17:23.235515", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-16T22:17:39.552 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:00.050) 0:01:13.501 ******* 2020-11-16T22:17:39.901 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:17:39.903 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-16T22:17:39.934 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:00.381) 0:01:13.883 ******* 2020-11-16T22:17:39.962 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-16T22:17:39.995 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:39 +0000 (0:00:00.061) 0:01:13.945 ******* 2020-11-16T22:17:40.148 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:17:40.149 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016603", "end": "2020-11-16 22:17:40.143659", "rc": 0, "start": "2020-11-16 22:17:40.127056", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:40.181 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:40 +0000 (0:00:00.185) 0:01:14.131 ******* 2020-11-16T22:17:40.199 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.009585", "end": "2020-11-16 22:17:40.193524", "rc": 0, "start": "2020-11-16 22:17:40.183939", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:40.231 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:40 +0000 (0:00:00.049) 0:01:14.181 ******* 2020-11-16T22:17:40.385 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:17:40.387 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.008014", "end": "2020-11-16 22:17:40.378236", "rc": 0, "start": "2020-11-16 22:17:40.370222", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:40.418 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:40 +0000 (0:00:00.186) 0:01:14.367 ******* 2020-11-16T22:17:40.436 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.007436", "end": "2020-11-16 22:17:40.425554", "rc": 0, "start": "2020-11-16 22:17:40.418118", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:40.471 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:40 +0000 (0:00:00.052) 0:01:14.420 ******* 2020-11-16T22:17:43.724 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:17:43.726 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-16T22:17:43.756 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:43 +0000 (0:00:03.284) 0:01:17.705 ******* 2020-11-16T22:17:43.960 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:17:43.962 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.011073", "end": "2020-11-16 22:17:43.947791", "rc": 0, "start": "2020-11-16 22:17:43.936718", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:43.991 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:43 +0000 (0:00:00.235) 0:01:17.941 ******* 2020-11-16T22:17:44.113 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:17:44.115 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-16T22:17:44.145 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:44 +0000 (0:00:00.153) 0:01:18.095 ******* 2020-11-16T22:17:44.193 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:17:44.195 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.010581", "end": "2020-11-16 22:17:44.189140", "rc": 0, "start": "2020-11-16 22:17:44.178559", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:44.225 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:44 +0000 (0:00:00.080) 0:01:18.175 ******* 2020-11-16T22:17:44.346 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:17:44.348 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.013338", "end": "2020-11-16 22:17:44.341833", "rc": 0, "start": "2020-11-16 22:17:44.328495", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:44.378 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:44 +0000 (0:00:00.152) 0:01:18.327 ******* 2020-11-16T22:17:44.571 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T22:17:44.573 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.007442", "end": "2020-11-16 22:17:44.568415", "rc": 0, "start": "2020-11-16 22:17:44.560973", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:44.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:44 +0000 (0:00:00.225) 0:01:18.552 ******* 2020-11-16T22:17:56.860 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-16T22:17:56.862 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.235860", "end": "2020-11-16 22:17:56.845461", "rc": 0, "start": "2020-11-16 22:17:40.609601", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:56.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:56 +0000 (0:00:12.290) 0:01:30.843 ******* 2020-11-16T22:17:56.964 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.292442", "end": "2020-11-16 22:17:56.955667", "rc": 0, "start": "2020-11-16 22:17:40.663225", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:17:56.997 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:17:56 +0000 (0:00:00.104) 0:01:30.947 ******* 2020-11-16T22:18:00.548 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.128057", "end": "2020-11-16 22:18:00.536406", "rc": 0, "start": "2020-11-16 22:17:44.408349", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:00.580 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:00 +0000 (0:00:03.582) 0:01:34.530 ******* 2020-11-16T22:18:01.123 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.324997", "end": "2020-11-16 22:18:01.110359", "rc": 0, "start": "2020-11-16 22:17:44.785362", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:01.155 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.575) 0:01:35.105 ******* 2020-11-16T22:18:01.214 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:18:01.216 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-16T22:18:01.247 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.091) 0:01:35.196 ******* 2020-11-16T22:18:01.275 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:18:01.277 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-16T22:18:01.306 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.059) 0:01:35.256 ******* 2020-11-16T22:18:01.323 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:18:01.325 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-16T22:18:01.355 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.048) 0:01:35.305 ******* 2020-11-16T22:18:01.361 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:18:01.363 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-16T22:18:01.395 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.345 ******* 2020-11-16T22:18:01.423 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.027) 0:01:35.373 ******* 2020-11-16T22:18:01.429 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:18:01.431 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:18:01.431 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"ena 2020-11-16T22:18:01.433 INFO:teuthology.task.ansible.out:bled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-16T22:18:01.464 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.413 ******* 2020-11-16T22:18:01.492 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.027) 0:01:35.441 ******* 2020-11-16T22:18:01.499 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:18:01.501 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} TASK [testnode : Including version specific variables.] ************************ ok: [smithi138.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:01.532 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.482 ******* 2020-11-16T22:18:01.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.029) 0:01:35.511 ******* 2020-11-16T22:18:01.568 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:18:01.569 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:01.602 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.551 ******* 2020-11-16T22:18:01.609 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:01.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.041) 0:01:35.592 ******* 2020-11-16T22:18:01.649 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:18:01.649 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-16T22:18:01.684 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.633 ******* 2020-11-16T22:18:01.691 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-16T22:18:01.725 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.041) 0:01:35.675 ******* 2020-11-16T22:18:01.731 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:18:01.732 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:18:01.765 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.038) 0:01:35.714 ******* 2020-11-16T22:18:01.793 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.028) 0:01:35.742 ******* 2020-11-16T22:18:01.799 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:18:01.801 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:01.833 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.040) 0:01:35.783 ******* 2020-11-16T22:18:01.954 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T22:18:01.955 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:18:01.987 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:01 +0000 (0:00:00.153) 0:01:35.936 ******* 2020-11-16T22:18:01.992 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:18:01.994 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:18:02.029 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.042) 0:01:35.979 ******* 2020-11-16T22:18:02.035 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-16T22:18:02.036 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:18:02.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.039) 0:01:36.018 ******* 2020-11-16T22:18:02.097 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.028) 0:01:36.047 ******* 2020-11-16T22:18:02.102 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:18:02.104 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:02.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.038) 0:01:36.085 ******* 2020-11-16T22:18:02.215 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T22:18:02.216 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:18:02.248 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.112) 0:01:36.197 ******* 2020-11-16T22:18:02.276 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-16T22:18:02.277 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:02.307 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.059) 0:01:36.257 ******* 2020-11-16T22:18:02.313 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T22:18:02.315 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:18:02.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.039) 0:01:36.296 ******* 2020-11-16T22:18:02.352 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:18:02.353 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:02.384 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.038) 0:01:36.334 ******* 2020-11-16T22:18:02.390 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-16T22:18:02.392 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:02.423 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.038) 0:01:36.372 ******* 2020-11-16T22:18:02.449 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:18:02.451 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:02.480 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.057) 0:01:36.430 ******* 2020-11-16T22:18:02.591 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:18:02.593 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-16T22:18:02.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.142) 0:01:36.576 ******* 2020-11-16T22:18:02.687 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-16T22:18:02.719 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:02 +0000 (0:00:00.092) 0:01:36.669 ******* 2020-11-16T22:18:04.934 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:18:04.936 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-16T22:18:04.967 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:04 +0000 (0:00:02.247) 0:01:38.916 ******* 2020-11-16T22:18:04.997 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:18:04.998 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-16T22:18:05.028 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.060) 0:01:38.977 ******* 2020-11-16T22:18:05.065 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:18:05.067 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:05.096 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.068) 0:01:39.045 ******* 2020-11-16T22:18:05.134 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T22:18:05.136 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-16T22:18:05.166 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.069) 0:01:39.115 ******* 2020-11-16T22:18:05.205 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T22:18:05.206 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T22:18:05.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.069) 0:01:39.185 ******* 2020-11-16T22:18:05.427 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:18:05.429 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:18:05.459 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.223) 0:01:39.408 ******* 2020-11-16T22:18:05.651 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T22:18:05.653 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:18:05.683 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.223) 0:01:39.632 ******* 2020-11-16T22:18:05.709 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-16T22:18:05.711 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:05.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.058) 0:01:39.690 ******* 2020-11-16T22:18:05.767 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T22:18:05.769 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:05.798 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:05 +0000 (0:00:00.057) 0:01:39.748 ******* 2020-11-16T22:18:06.001 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:18:06.003 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-16T22:18:06.034 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:06 +0000 (0:00:00.236) 0:01:39.984 ******* 2020-11-16T22:18:06.498 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-16T22:18:06.530 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:06 +0000 (0:00:00.495) 0:01:40.480 ******* 2020-11-16T22:18:06.537 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:18:06.539 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:18:06.571 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:06 +0000 (0:00:00.040) 0:01:40.521 ******* 2020-11-16T22:18:06.599 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:18:06.601 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi122"}, "changed": false} 2020-11-16T22:18:06.631 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:06 +0000 (0:00:00.059) 0:01:40.580 ******* 2020-11-16T22:18:11.216 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:18:11.218 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:18:11.249 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:04.618) 0:01:45.199 ******* 2020-11-16T22:18:11.276 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:18:11.278 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi138"}, "changed": false} 2020-11-16T22:18:11.307 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.057) 0:01:45.256 ******* 2020-11-16T22:18:11.521 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:18:11.523 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:18:11.553 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.246) 0:01:45.503 ******* 2020-11-16T22:18:11.581 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:18:11.582 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:18:11.582 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi173"}, "changed": false} 2020-11-16T22:18:11.613 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.059) 0:01:45.562 ******* 2020-11-16T22:18:11.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.030) 0:01:45.593 ******* 2020-11-16T22:18:11.671 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi194"}, "changed": false} 2020-11-16T22:18:11.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.058) 0:01:45.651 ******* 2020-11-16T22:18:11.730 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:18:11.731 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi122", "ansible_hostname": "smithi122", "ansible_nodename": "smithi122"}, "changed": false, "name": "smithi122"} 2020-11-16T22:18:11.761 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.059) 0:01:45.711 ******* 2020-11-16T22:18:11.873 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi138", "ansible_hostname": "smithi138", "ansible_nodename": "smithi138"}, "changed": false, "name": "smithi138"} 2020-11-16T22:18:11.904 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:11 +0000 (0:00:00.142) 0:01:45.854 ******* 2020-11-16T22:18:12.160 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi173", "ansible_hostname": "smithi173", "ansible_nodename": "smithi173"}, "changed": false, "name": "smithi173"} 2020-11-16T22:18:12.195 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.290) 0:01:46.144 ******* 2020-11-16T22:18:12.212 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:18:12.214 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-16T22:18:12.245 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.049) 0:01:46.194 ******* 2020-11-16T22:18:12.262 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:18:12.264 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi194", "ansible_hostname": "smithi194", "ansible_nodename": "smithi194"}, "changed": false, "name": "smithi194"} 2020-11-16T22:18:12.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.052) 0:01:46.247 ******* 2020-11-16T22:18:12.346 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:18:12.348 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-16T22:18:12.379 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.081) 0:01:46.328 ******* 2020-11-16T22:18:12.647 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-16T22:18:12.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.300) 0:01:46.629 ******* 2020-11-16T22:18:12.696 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:18:12.698 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:18:12.731 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.051) 0:01:46.681 ******* 2020-11-16T22:18:12.749 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:18:12.750 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-16T22:18:12.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.050) 0:01:46.732 ******* 2020-11-16T22:18:12.820 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:18:12.821 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:18:12.855 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:12 +0000 (0:00:00.073) 0:01:46.805 ******* 2020-11-16T22:18:13.122 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:18:13.155 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.299) 0:01:47.104 ******* 2020-11-16T22:18:13.235 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:18:13.268 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.112) 0:01:47.217 ******* 2020-11-16T22:18:13.451 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:18:13.453 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:18:13.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.215) 0:01:47.433 ******* 2020-11-16T22:18:13.490 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:18:13.523 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.039) 0:01:47.472 ******* 2020-11-16T22:18:13.768 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:18:13.800 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.277) 0:01:47.750 ******* 2020-11-16T22:18:13.870 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:18:13.902 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.101) 0:01:47.852 ******* 2020-11-16T22:18:13.940 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:18:13.942 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-16T22:18:13.978 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:13 +0000 (0:00:00.075) 0:01:47.927 ******* 2020-11-16T22:18:13.986 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-16T22:18:14.018 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.040) 0:01:47.968 ******* 2020-11-16T22:18:14.172 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:18:14.174 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:18:14.204 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.185) 0:01:48.153 ******* 2020-11-16T22:18:14.222 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:18:14.255 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.050) 0:01:48.204 ******* 2020-11-16T22:18:14.271 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:18:14.273 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-16T22:18:14.304 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.049) 0:01:48.253 ******* 2020-11-16T22:18:14.373 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-16T22:18:14.406 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.102) 0:01:48.356 ******* 2020-11-16T22:18:14.412 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:18:14.413 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-16T22:18:14.446 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.039) 0:01:48.395 ******* 2020-11-16T22:18:14.452 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-16T22:18:14.486 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.040) 0:01:48.435 ******* 2020-11-16T22:18:14.503 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:18:14.505 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:18:14.536 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.050) 0:01:48.486 ******* 2020-11-16T22:18:14.605 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:18:14.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.102) 0:01:48.588 ******* 2020-11-16T22:18:14.666 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:18:14.668 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:18:14.699 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.060) 0:01:48.649 ******* 2020-11-16T22:18:14.707 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:18:14.743 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.043) 0:01:48.693 ******* 2020-11-16T22:18:14.750 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:18:14.751 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-16T22:18:14.784 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.040) 0:01:48.733 ******* 2020-11-16T22:18:14.836 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-16T22:18:14.871 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:14 +0000 (0:00:00.084) 0:01:48.818 ******* 2020-11-16T22:18:15.000 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:18:15.002 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:18:15.034 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.165) 0:01:48.983 ******* 2020-11-16T22:18:15.082 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:18:15.114 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.080) 0:01:49.064 ******* 2020-11-16T22:18:15.172 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:18:15.173 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-16T22:18:15.210 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.095) 0:01:49.159 ******* 2020-11-16T22:18:15.217 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-16T22:18:15.249 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.039) 0:01:49.199 ******* 2020-11-16T22:18:15.497 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-16T22:18:15.529 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.279) 0:01:49.479 ******* 2020-11-16T22:18:15.577 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-16T22:18:15.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.080) 0:01:49.559 ******* 2020-11-16T22:18:15.667 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:18:15.669 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-16T22:18:15.703 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.093) 0:01:49.652 ******* 2020-11-16T22:18:15.711 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-16T22:18:15.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:15 +0000 (0:00:00.040) 0:01:49.693 ******* 2020-11-16T22:18:15.989 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-16T22:18:16.022 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:16 +0000 (0:00:00.278) 0:01:49.971 ******* 2020-11-16T22:18:16.070 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-16T22:18:16.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:16 +0000 (0:00:00.080) 0:01:50.052 ******* 2020-11-16T22:18:17.656 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:18:17.658 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.756237", "end": "2020-11-16 22:18:17.640636", "rc": 0, "start": "2020-11-16 22:18:15.884399", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.717889", "end": "2020-11-16 22:18:17.642293", "rc": 0, "start": "2020-11-16 22:18:15.924404", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:17.689 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:17 +0000 (0:00:01.586) 0:01:51.639 ******* 2020-11-16T22:18:17.719 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:17 +0000 (0:00:00.029) 0:01:51.669 ******* 2020-11-16T22:18:17.845 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.551687", "end": "2020-11-16 22:18:17.835902", "rc": 0, "start": "2020-11-16 22:18:16.284215", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:17.876 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:17 +0000 (0:00:00.156) 0:01:51.825 ******* 2020-11-16T22:18:17.883 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.656406", "end": "2020-11-16 22:18:17.857561", "rc": 0, "start": "2020-11-16 22:18:16.201155", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:17.917 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:17 +0000 (0:00:00.041) 0:01:51.867 ******* 2020-11-16T22:18:18.175 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T22:18:18.176 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-16T22:18:18.207 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-16T22:18:18.355 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-16T22:18:18.398 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-16T22:18:18.607 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-16T22:18:18.639 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-16T22:18:18.787 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-16T22:18:18.829 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-16T22:18:19.039 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-16T22:18:19.074 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:01.155) 0:01:53.023 ******* 2020-11-16T22:18:19.080 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-16T22:18:19.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.042) 0:01:53.065 ******* 2020-11-16T22:18:19.216 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-16T22:18:19.251 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.135) 0:01:53.200 ******* 2020-11-16T22:18:19.269 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-16T22:18:19.304 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.053) 0:01:53.253 ******* 2020-11-16T22:18:19.540 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:18:19.542 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-16T22:18:19.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.270) 0:01:53.524 ******* 2020-11-16T22:18:19.583 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-16T22:18:19.617 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.042) 0:01:53.566 ******* 2020-11-16T22:18:19.719 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-16T22:18:19.752 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.134) 0:01:53.701 ******* 2020-11-16T22:18:19.770 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-16T22:18:19.803 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:19 +0000 (0:00:00.050) 0:01:53.752 ******* 2020-11-16T22:18:20.043 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:18:20.044 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-16T22:18:20.086 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-16T22:18:20.222 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-16T22:18:20.285 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-16T22:18:20.485 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-16T22:18:20.517 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-16T22:18:20.664 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-16T22:18:20.718 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-16T22:18:20.916 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-16T22:18:20.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:20 +0000 (0:00:01.148) 0:01:54.900 ******* 2020-11-16T22:18:20.959 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-16T22:18:20.998 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:20 +0000 (0:00:00.047) 0:01:54.948 ******* 2020-11-16T22:18:21.006 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:18:21.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.038) 0:01:54.987 ******* 2020-11-16T22:18:21.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.039) 0:01:55.026 ******* 2020-11-16T22:18:21.084 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:18:21.085 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.118 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.041) 0:01:55.068 ******* 2020-11-16T22:18:21.129 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:18:21.130 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-16T22:18:21.136 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:18:21.137 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.182 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.064) 0:01:55.132 ******* 2020-11-16T22:18:21.211 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.028) 0:01:55.160 ******* 2020-11-16T22:18:21.218 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:18:21.220 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.221 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:18:21.222 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-16T22:18:21.225 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:18:21.226 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.257 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.046) 0:01:55.207 ******* 2020-11-16T22:18:21.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.029) 0:01:55.236 ******* 2020-11-16T22:18:21.315 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.028) 0:01:55.265 ******* 2020-11-16T22:18:21.322 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:18:21.323 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.354 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.038) 0:01:55.303 ******* 2020-11-16T22:18:21.382 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.028) 0:01:55.332 ******* 2020-11-16T22:18:21.412 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.030) 0:01:55.362 ******* 2020-11-16T22:18:21.418 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:18:21.419 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:18:21.419 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-16T22:18:21.421 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-16T22:18:21.452 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.039) 0:01:55.401 ******* 2020-11-16T22:18:21.482 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.029) 0:01:55.431 ******* 2020-11-16T22:18:21.511 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.028) 0:01:55.460 ******* 2020-11-16T22:18:21.517 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:18:21.519 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.551 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.040) 0:01:55.500 ******* 2020-11-16T22:18:21.580 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.029) 0:01:55.530 ******* 2020-11-16T22:18:21.611 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.030) 0:01:55.560 ******* 2020-11-16T22:18:21.616 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:18:21.618 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.040) 0:01:55.600 ******* 2020-11-16T22:18:21.678 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.027) 0:01:55.628 ******* 2020-11-16T22:18:21.709 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.030) 0:01:55.658 ******* 2020-11-16T22:18:21.715 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:18:21.717 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.749 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.041) 0:01:55.699 ******* 2020-11-16T22:18:21.755 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.757 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.790 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.039) 0:01:55.739 ******* 2020-11-16T22:18:21.796 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:18:21.798 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.830 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.040) 0:01:55.779 ******* 2020-11-16T22:18:21.836 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.869 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.039) 0:01:55.819 ******* 2020-11-16T22:18:21.876 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:18:21.876 INFO:teuthology.task.ansible.out:fatal: [smithi122.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006548", "end": "2020-11-16 22:18:21.842802", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:18:21.836254", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-16T22:18:21.878 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-16T22:18:21.910 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.040) 0:01:55.860 ******* 2020-11-16T22:18:21.939 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.029) 0:01:55.889 ******* 2020-11-16T22:18:21.945 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:18:21.946 INFO:teuthology.task.ansible.out:fatal: [smithi138.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006689", "end": "2020-11-16 22:18:21.880159", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:18:21.873470", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-16T22:18:21.948 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:21.980 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:21 +0000 (0:00:00.040) 0:01:55.929 ******* 2020-11-16T22:18:22.009 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.029) 0:01:55.958 ******* 2020-11-16T22:18:22.039 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.029) 0:01:55.989 ******* 2020-11-16T22:18:22.047 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:18:22.048 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.080 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.041) 0:01:56.030 ******* 2020-11-16T22:18:22.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.029) 0:01:56.059 ******* 2020-11-16T22:18:22.116 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:18:22.118 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.156 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.046) 0:01:56.105 ******* 2020-11-16T22:18:22.163 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T22:18:22.165 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.196 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.039) 0:01:56.145 ******* 2020-11-16T22:18:22.203 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.236 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.040) 0:01:56.186 ******* 2020-11-16T22:18:22.411 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:18:22.413 INFO:teuthology.task.ansible.out:fatal: [smithi194.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006953", "end": "2020-11-16 22:18:22.397987", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:18:22.391034", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-16T22:18:22.444 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.207) 0:01:56.393 ******* 2020-11-16T22:18:22.451 INFO:teuthology.task.ansible.out:fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006625", "end": "2020-11-16 22:18:22.440234", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:18:22.433609", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-16T22:18:22.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.039) 0:01:56.433 ******* 2020-11-16T22:18:22.489 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T22:18:22.491 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.523 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.039) 0:01:56.472 ******* 2020-11-16T22:18:22.529 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:22.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:22 +0000 (0:00:00.039) 0:01:56.511 ******* 2020-11-16T22:18:24.180 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:18:24.182 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-16T22:18:24.213 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:24 +0000 (0:00:01.650) 0:01:58.162 ******* 2020-11-16T22:18:24.220 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-16T22:18:24.256 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:24 +0000 (0:00:00.043) 0:01:58.205 ******* 2020-11-16T22:18:24.589 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-16T22:18:24.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:24 +0000 (0:00:00.370) 0:01:58.576 ******* 2020-11-16T22:18:24.770 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-16T22:18:24.942 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:24 +0000 (0:00:00.314) 0:01:58.891 ******* 2020-11-16T22:18:29.149 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:18:29.151 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:29.152 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:29.185 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:29 +0000 (0:00:04.243) 0:02:03.135 ******* 2020-11-16T22:18:29.220 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:29 +0000 (0:00:00.034) 0:02:03.169 ******* 2020-11-16T22:18:29.256 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:29 +0000 (0:00:00.036) 0:02:03.206 ******* 2020-11-16T22:18:31.102 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:31.135 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:31 +0000 (0:00:01.878) 0:02:05.084 ******* 2020-11-16T22:18:32.147 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:32.148 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:32.179 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:01.044) 0:02:06.129 ******* 2020-11-16T22:18:32.212 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:18:32.213 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:32.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:00.063) 0:02:06.192 ******* 2020-11-16T22:18:32.249 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:32.249 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:32.251 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:32.285 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:00.041) 0:02:06.234 ******* 2020-11-16T22:18:32.316 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:00.030) 0:02:06.265 ******* 2020-11-16T22:18:32.321 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:18:32.323 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:32.354 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:00.038) 0:02:06.303 ******* 2020-11-16T22:18:32.360 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:32.393 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:32 +0000 (0:00:00.039) 0:02:06.342 ******* 2020-11-16T22:18:33.159 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:18:33.161 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-16T22:18:33.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.799) 0:02:07.142 ******* 2020-11-16T22:18:33.220 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:33.222 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:33.251 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.058) 0:02:07.200 ******* 2020-11-16T22:18:33.300 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:18:33.302 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-16T22:18:33.333 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.082) 0:02:07.283 ******* 2020-11-16T22:18:33.341 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-16T22:18:33.375 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.041) 0:02:07.324 ******* 2020-11-16T22:18:33.382 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:33.383 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:33.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.040) 0:02:07.364 ******* 2020-11-16T22:18:33.422 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:33.455 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.040) 0:02:07.405 ******* 2020-11-16T22:18:33.556 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:18:33.557 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service basic.target dbus.socket system.slice sysinit.target polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ip6tables.service ebtables.service shutdown.target iptables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-16T22:18:33.558 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", 2020-11-16T22:18:33.560 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket system.slice sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-16T22:18:33.561 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-16T22:18:33.590 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.134) 0:02:07.539 ******* 2020-11-16T22:18:33.727 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service dbus.socket system.slice polkit.service sysinit.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "iptables.service shutdown.target ipset.service ip6tables.service ebtables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-16T22:18:33.729 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", 2020-11-16T22:18:33.731 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket system.slice sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-16T22:18:33.732 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-16T22:18:33.759 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.168) 0:02:07.708 ******* 2020-11-16T22:18:33.768 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.socket basic.target polkit.service dbus.service sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ip6tables.service shutdown.target ebtables.service iptables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-16T22:18:33.769 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", 2020-11-16T22:18:33.771 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket sysinit.target system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-16T22:18:33.772 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-16T22:18:33.801 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.042) 0:02:07.751 ******* 2020-11-16T22:18:33.881 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:18:33.882 INFO:teuthology.task.ansible.out:fatal: [smithi138.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-16T22:18:33.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:33 +0000 (0:00:00.111) 0:02:07.863 ******* 2020-11-16T22:18:34.045 INFO:teuthology.task.ansible.out:fatal: [smithi122.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-16T22:18:34.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.163) 0:02:08.027 ******* 2020-11-16T22:18:34.095 INFO:teuthology.task.ansible.out:fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-16T22:18:34.127 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.050) 0:02:08.077 ******* 2020-11-16T22:18:34.135 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:34.137 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-16T22:18:34.169 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.041) 0:02:08.118 ******* 2020-11-16T22:18:34.196 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:18:34.198 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.228 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.058) 0:02:08.177 ******* 2020-11-16T22:18:34.452 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:18:34.454 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-16T22:18:34.486 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.257) 0:02:08.435 ******* 2020-11-16T22:18:34.515 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.029) 0:02:08.465 ******* 2020-11-16T22:18:34.543 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.027) 0:02:08.493 ******* 2020-11-16T22:18:34.548 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:18:34.550 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.037) 0:02:08.531 ******* 2020-11-16T22:18:34.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.027) 0:02:08.558 ******* 2020-11-16T22:18:34.615 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.616 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.646 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.037) 0:02:08.596 ******* 2020-11-16T22:18:34.675 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.028) 0:02:08.624 ******* 2020-11-16T22:18:34.680 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.682 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.713 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.037) 0:02:08.662 ******* 2020-11-16T22:18:34.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.028) 0:02:08.690 ******* 2020-11-16T22:18:34.748 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:18:34.750 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.784 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.043) 0:02:08.734 ******* 2020-11-16T22:18:34.813 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.028) 0:02:08.762 ******* 2020-11-16T22:18:34.820 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:18:34.822 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.859 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.046) 0:02:08.808 ******* 2020-11-16T22:18:34.888 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.029) 0:02:08.838 ******* 2020-11-16T22:18:34.894 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:18:34.896 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:34.928 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.039) 0:02:08.877 ******* 2020-11-16T22:18:34.956 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.028) 0:02:08.905 ******* 2020-11-16T22:18:34.962 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:18:34.964 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:18:34.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:34 +0000 (0:00:00.038) 0:02:08.944 ******* 2020-11-16T22:18:35.024 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:08.974 ******* 2020-11-16T22:18:35.031 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:18:35.033 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.064 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.040) 0:02:09.014 ******* 2020-11-16T22:18:35.094 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.044 ******* 2020-11-16T22:18:35.100 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:18:35.101 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.133 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.038) 0:02:09.082 ******* 2020-11-16T22:18:35.162 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.112 ******* 2020-11-16T22:18:35.168 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:18:35.170 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-16T22:18:35.170 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.172 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.202 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.040) 0:02:09.152 ******* 2020-11-16T22:18:35.231 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.028) 0:02:09.181 ******* 2020-11-16T22:18:35.260 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.028) 0:02:09.209 ******* 2020-11-16T22:18:35.266 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:18:35.268 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-apt] ******************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.268 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.304 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.043) 0:02:09.253 ******* 2020-11-16T22:18:35.339 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.034) 0:02:09.288 ******* 2020-11-16T22:18:35.375 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.035) 0:02:09.324 ******* 2020-11-16T22:18:35.381 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:35.382 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add local apt repos.] ***************************************** TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:18:35.413 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.038) 0:02:09.362 ******* 2020-11-16T22:18:35.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.028) 0:02:09.391 ******* 2020-11-16T22:18:35.474 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.031) 0:02:09.423 ******* 2020-11-16T22:18:35.479 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:18:35.481 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.516 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.042) 0:02:09.466 ******* 2020-11-16T22:18:35.547 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.030) 0:02:09.496 ******* 2020-11-16T22:18:35.577 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.526 ******* 2020-11-16T22:18:35.582 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:18:35.584 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.615 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.038) 0:02:09.565 ******* 2020-11-16T22:18:35.645 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.594 ******* 2020-11-16T22:18:35.652 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:18:35.654 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.686 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.041) 0:02:09.635 ******* 2020-11-16T22:18:35.716 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.665 ******* 2020-11-16T22:18:35.723 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:18:35.724 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.725 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.756 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.040) 0:02:09.706 ******* 2020-11-16T22:18:35.786 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.030) 0:02:09.736 ******* 2020-11-16T22:18:35.793 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:18:35.795 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:18:35.796 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.socket system.slice dbus.service sysinit.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ip6tables.service shutdown.target iptables.service ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-16T22:18:35.798 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", 2020-11-16T22:18:35.799 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket system.slice sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-16T22:18:35.801 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.829 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.042) 0:02:09.778 ******* 2020-11-16T22:18:35.859 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.030) 0:02:09.808 ******* 2020-11-16T22:18:35.888 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.028) 0:02:09.837 ******* 2020-11-16T22:18:35.894 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:18:35.895 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* skipping: [smithi173.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.895 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:35.928 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.040) 0:02:09.878 ******* 2020-11-16T22:18:35.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.908 ******* 2020-11-16T22:18:35.987 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:35 +0000 (0:00:00.029) 0:02:09.937 ******* 2020-11-16T22:18:35.994 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T22:18:35.995 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:18:36.027 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.039) 0:02:09.977 ******* 2020-11-16T22:18:36.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.029) 0:02:10.006 ******* 2020-11-16T22:18:36.062 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:18:36.064 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.095 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.038) 0:02:10.045 ******* 2020-11-16T22:18:36.123 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.073 ******* 2020-11-16T22:18:36.129 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:18:36.131 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.166 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.042) 0:02:10.116 ******* 2020-11-16T22:18:36.195 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.144 ******* 2020-11-16T22:18:36.201 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:18:36.202 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi194.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.234 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.039) 0:02:10.183 ******* 2020-11-16T22:18:36.262 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.212 ******* 2020-11-16T22:18:36.291 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.240 ******* 2020-11-16T22:18:36.296 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:18:36.298 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.039) 0:02:10.280 ******* 2020-11-16T22:18:36.364 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.032) 0:02:10.313 ******* 2020-11-16T22:18:36.394 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.030) 0:02:10.343 ******* 2020-11-16T22:18:36.400 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:18:36.402 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.403 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.447 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.052) 0:02:10.396 ******* 2020-11-16T22:18:36.478 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.031) 0:02:10.427 ******* 2020-11-16T22:18:36.484 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:18:36.486 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.516 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.038) 0:02:10.466 ******* 2020-11-16T22:18:36.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.029) 0:02:10.495 ******* 2020-11-16T22:18:36.551 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:36.554 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop apache2] ************************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.584 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.039) 0:02:10.534 ******* 2020-11-16T22:18:36.613 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.562 ******* 2020-11-16T22:18:36.618 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:18:36.620 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.038) 0:02:10.601 ******* 2020-11-16T22:18:36.680 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.629 ******* 2020-11-16T22:18:36.708 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.657 ******* 2020-11-16T22:18:36.713 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:18:36.715 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.038) 0:02:10.696 ******* 2020-11-16T22:18:36.775 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.724 ******* 2020-11-16T22:18:36.803 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.753 ******* 2020-11-16T22:18:36.809 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:18:36.811 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.841 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.037) 0:02:10.791 ******* 2020-11-16T22:18:36.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.032) 0:02:10.823 ******* 2020-11-16T22:18:36.904 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.031) 0:02:10.854 ******* 2020-11-16T22:18:36.910 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:18:36.912 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:36.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.039) 0:02:10.893 ******* 2020-11-16T22:18:36.972 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.028) 0:02:10.922 ******* 2020-11-16T22:18:37.002 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:36 +0000 (0:00:00.029) 0:02:10.951 ******* 2020-11-16T22:18:37.009 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:18:37.011 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.011 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:18:37.011 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.044 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.041) 0:02:10.993 ******* 2020-11-16T22:18:37.073 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.022 ******* 2020-11-16T22:18:37.101 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.028) 0:02:11.051 ******* 2020-11-16T22:18:37.108 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:37.109 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:18:37.111 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.040) 0:02:11.091 ******* 2020-11-16T22:18:37.171 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.028) 0:02:11.120 ******* 2020-11-16T22:18:37.206 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.035) 0:02:11.156 ******* 2020-11-16T22:18:37.214 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:18:37.216 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.216 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.249 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.043) 0:02:11.199 ******* 2020-11-16T22:18:37.285 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.035) 0:02:11.235 ******* 2020-11-16T22:18:37.315 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.264 ******* 2020-11-16T22:18:37.322 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T22:18:37.323 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.042) 0:02:11.306 ******* 2020-11-16T22:18:37.388 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.031) 0:02:11.338 ******* 2020-11-16T22:18:37.419 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.030) 0:02:11.368 ******* 2020-11-16T22:18:37.424 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:18:37.426 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.038) 0:02:11.407 ******* 2020-11-16T22:18:37.487 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.436 ******* 2020-11-16T22:18:37.516 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.028) 0:02:11.465 ******* 2020-11-16T22:18:37.522 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:18:37.524 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.555 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.039) 0:02:11.504 ******* 2020-11-16T22:18:37.587 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.032) 0:02:11.537 ******* 2020-11-16T22:18:37.617 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.028) 0:02:11.566 ******* 2020-11-16T22:18:37.622 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:18:37.624 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop apache2] ************************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.039) 0:02:11.606 ******* 2020-11-16T22:18:37.685 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.028) 0:02:11.634 ******* 2020-11-16T22:18:37.714 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.663 ******* 2020-11-16T22:18:37.720 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T22:18:37.721 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.038) 0:02:11.702 ******* 2020-11-16T22:18:37.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.732 ******* 2020-11-16T22:18:37.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.029) 0:02:11.761 ******* 2020-11-16T22:18:37.819 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:18:37.821 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.852 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.039) 0:02:11.801 ******* 2020-11-16T22:18:37.894 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.041) 0:02:11.843 ******* 2020-11-16T22:18:37.930 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.036) 0:02:11.880 ******* 2020-11-16T22:18:37.936 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:18:37.938 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:37.970 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.039) 0:02:11.919 ******* 2020-11-16T22:18:38.005 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:37 +0000 (0:00:00.035) 0:02:11.955 ******* 2020-11-16T22:18:38.013 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:18:38.013 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:18:38.014 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.047 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.041) 0:02:11.996 ******* 2020-11-16T22:18:38.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.035) 0:02:12.031 ******* 2020-11-16T22:18:38.087 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:18:38.089 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-16T22:18:38.089 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.039) 0:02:12.071 ******* 2020-11-16T22:18:38.152 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.030) 0:02:12.101 ******* 2020-11-16T22:18:38.181 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.029) 0:02:12.131 ******* 2020-11-16T22:18:38.187 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:18:38.189 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.219 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.038) 0:02:12.169 ******* 2020-11-16T22:18:38.248 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.028) 0:02:12.198 ******* 2020-11-16T22:18:38.255 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:18:38.256 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.289 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.040) 0:02:12.239 ******* 2020-11-16T22:18:38.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.033) 0:02:12.272 ******* 2020-11-16T22:18:38.329 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:18:38.330 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.330 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:18:38.365 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.042) 0:02:12.314 ******* 2020-11-16T22:18:38.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.031) 0:02:12.345 ******* 2020-11-16T22:18:38.425 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.029) 0:02:12.375 ******* 2020-11-16T22:18:38.431 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:18:38.432 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.465 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.039) 0:02:12.415 ******* 2020-11-16T22:18:38.494 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.029) 0:02:12.444 ******* 2020-11-16T22:18:38.501 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:18:38.502 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.504 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.536 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.041) 0:02:12.485 ******* 2020-11-16T22:18:38.566 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.030) 0:02:12.516 ******* 2020-11-16T22:18:38.573 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:18:38.575 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.576 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-16T22:18:38.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.042) 0:02:12.559 ******* 2020-11-16T22:18:38.644 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.035) 0:02:12.594 ******* 2020-11-16T22:18:38.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.035) 0:02:12.629 ******* 2020-11-16T22:18:38.685 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:18:38.687 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.719 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.039) 0:02:12.668 ******* 2020-11-16T22:18:38.755 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.035) 0:02:12.704 ******* 2020-11-16T22:18:38.791 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.036) 0:02:12.741 ******* 2020-11-16T22:18:38.797 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:18:38.798 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.829 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.037) 0:02:12.779 ******* 2020-11-16T22:18:38.858 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.028) 0:02:12.807 ******* 2020-11-16T22:18:38.887 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.028) 0:02:12.836 ******* 2020-11-16T22:18:38.894 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:18:38.896 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:38.930 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.043) 0:02:12.880 ******* 2020-11-16T22:18:38.960 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.030) 0:02:12.910 ******* 2020-11-16T22:18:38.967 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:18:38.968 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.000 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:38 +0000 (0:00:00.039) 0:02:12.949 ******* 2020-11-16T22:18:39.029 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:12.978 ******* 2020-11-16T22:18:39.035 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:18:39.036 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.072 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.043) 0:02:13.022 ******* 2020-11-16T22:18:39.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.051 ******* 2020-11-16T22:18:39.107 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:18:39.109 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-16T22:18:39.109 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:39.109 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.040) 0:02:13.092 ******* 2020-11-16T22:18:39.172 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.121 ******* 2020-11-16T22:18:39.201 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.151 ******* 2020-11-16T22:18:39.209 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:18:39.210 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.241 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.039) 0:02:13.191 ******* 2020-11-16T22:18:39.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.220 ******* 2020-11-16T22:18:39.276 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:18:39.278 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.309 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.038) 0:02:13.258 ******* 2020-11-16T22:18:39.339 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.288 ******* 2020-11-16T22:18:39.345 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T22:18:39.346 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.377 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.038) 0:02:13.327 ******* 2020-11-16T22:18:39.407 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.030) 0:02:13.357 ******* 2020-11-16T22:18:39.414 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:18:39.415 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:39.416 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:18:39.448 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.040) 0:02:13.397 ******* 2020-11-16T22:18:39.477 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.427 ******* 2020-11-16T22:18:39.507 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.456 ******* 2020-11-16T22:18:39.513 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:39.515 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.038) 0:02:13.495 ******* 2020-11-16T22:18:39.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.524 ******* 2020-11-16T22:18:39.581 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:18:39.583 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.039) 0:02:13.563 ******* 2020-11-16T22:18:39.642 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.592 ******* 2020-11-16T22:18:39.648 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:18:39.649 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.680 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.037) 0:02:13.630 ******* 2020-11-16T22:18:39.708 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.658 ******* 2020-11-16T22:18:39.713 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:18:39.715 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-16T22:18:39.715 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.038) 0:02:13.696 ******* 2020-11-16T22:18:39.777 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.030) 0:02:13.727 ******* 2020-11-16T22:18:39.805 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.755 ******* 2020-11-16T22:18:39.811 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:39.812 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.844 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.038) 0:02:13.793 ******* 2020-11-16T22:18:39.872 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.821 ******* 2020-11-16T22:18:39.900 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.850 ******* 2020-11-16T22:18:39.906 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:18:39.907 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:39.938 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.037) 0:02:13.887 ******* 2020-11-16T22:18:39.966 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.028) 0:02:13.916 ******* 2020-11-16T22:18:39.996 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:39 +0000 (0:00:00.029) 0:02:13.945 ******* 2020-11-16T22:18:40.004 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T22:18:40.006 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.036 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.040) 0:02:13.986 ******* 2020-11-16T22:18:40.067 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.030) 0:02:14.017 ******* 2020-11-16T22:18:40.097 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.046 ******* 2020-11-16T22:18:40.104 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:18:40.105 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.041) 0:02:14.088 ******* 2020-11-16T22:18:40.166 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.027) 0:02:14.116 ******* 2020-11-16T22:18:40.196 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.145 ******* 2020-11-16T22:18:40.202 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:18:40.204 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.039) 0:02:14.184 ******* 2020-11-16T22:18:40.265 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.214 ******* 2020-11-16T22:18:40.294 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.243 ******* 2020-11-16T22:18:40.301 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:18:40.303 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.335 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.041) 0:02:14.285 ******* 2020-11-16T22:18:40.367 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.031) 0:02:14.316 ******* 2020-11-16T22:18:40.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.346 ******* 2020-11-16T22:18:40.402 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T22:18:40.403 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.435 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.038) 0:02:14.384 ******* 2020-11-16T22:18:40.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.028) 0:02:14.413 ******* 2020-11-16T22:18:40.499 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.036) 0:02:14.449 ******* 2020-11-16T22:18:40.506 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:40.508 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.539 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.039) 0:02:14.488 ******* 2020-11-16T22:18:40.568 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.517 ******* 2020-11-16T22:18:40.598 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.030) 0:02:14.547 ******* 2020-11-16T22:18:40.604 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:18:40.605 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.039) 0:02:14.587 ******* 2020-11-16T22:18:40.667 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.616 ******* 2020-11-16T22:18:40.698 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.030) 0:02:14.647 ******* 2020-11-16T22:18:40.705 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:18:40.706 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.707 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} ok: [smithi122.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:18:40.708 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.708 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.708 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi122.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.708 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.709 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.740 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.042) 0:02:14.689 ******* 2020-11-16T22:18:40.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.028) 0:02:14.718 ******* 2020-11-16T22:18:40.775 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.777 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:18:40.778 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.044) 0:02:14.762 ******* 2020-11-16T22:18:40.841 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.028) 0:02:14.790 ******* 2020-11-16T22:18:40.870 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.029) 0:02:14.820 ******* 2020-11-16T22:18:40.877 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:18:40.878 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.910 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.040) 0:02:14.860 ******* 2020-11-16T22:18:40.941 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.030) 0:02:14.891 ******* 2020-11-16T22:18:40.947 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:18:40.949 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:40.983 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:40 +0000 (0:00:00.041) 0:02:14.933 ******* 2020-11-16T22:18:41.014 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.031) 0:02:14.964 ******* 2020-11-16T22:18:41.021 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T22:18:41.023 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003560", "end": "2020-11-16 22:18:40.973408", "rc": 0, "start": "2020-11-16 22:18:40.969848", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.058 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.043) 0:02:15.007 ******* 2020-11-16T22:18:41.087 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.037 ******* 2020-11-16T22:18:41.095 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:18:41.096 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.048) 0:02:15.085 ******* 2020-11-16T22:18:41.165 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.115 ******* 2020-11-16T22:18:41.172 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T22:18:41.174 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-16T22:18:41.174 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.207 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.041) 0:02:15.157 ******* 2020-11-16T22:18:41.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.028) 0:02:15.185 ******* 2020-11-16T22:18:41.265 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.215 ******* 2020-11-16T22:18:41.273 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:18:41.274 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.306 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.256 ******* 2020-11-16T22:18:41.336 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.285 ******* 2020-11-16T22:18:41.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.030) 0:02:15.315 ******* 2020-11-16T22:18:41.371 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:18:41.373 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.405 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.038) 0:02:15.354 ******* 2020-11-16T22:18:41.435 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.384 ******* 2020-11-16T22:18:41.441 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:18:41.443 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.475 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.425 ******* 2020-11-16T22:18:41.482 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:18:41.483 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:18:41.515 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.465 ******* 2020-11-16T22:18:41.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.494 ******* 2020-11-16T22:18:41.550 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:18:41.552 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.584 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.039) 0:02:15.533 ******* 2020-11-16T22:18:41.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.563 ******* 2020-11-16T22:18:41.621 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:18:41.623 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.654 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.603 ******* 2020-11-16T22:18:41.660 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:18:41.662 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.693 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.038) 0:02:15.642 ******* 2020-11-16T22:18:41.698 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T22:18:41.700 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.037) 0:02:15.680 ******* 2020-11-16T22:18:41.737 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:18:41.739 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.740 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:41.740 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-16T22:18:41.771 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.721 ******* 2020-11-16T22:18:41.800 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.029) 0:02:15.750 ******* 2020-11-16T22:18:41.806 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T22:18:41.808 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.037) 0:02:15.788 ******* 2020-11-16T22:18:41.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.034) 0:02:15.822 ******* 2020-11-16T22:18:41.879 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:18:41.880 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:18:41.911 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.037) 0:02:15.860 ******* 2020-11-16T22:18:41.939 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.028) 0:02:15.889 ******* 2020-11-16T22:18:41.946 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:18:41.947 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi138.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.980 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:41 +0000 (0:00:00.040) 0:02:15.929 ******* 2020-11-16T22:18:41.986 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:18:41.987 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.988 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.988 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:18:41.988 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.989 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.989 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:41.989 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.024 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.044) 0:02:15.974 ******* 2020-11-16T22:18:42.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.027) 0:02:16.002 ******* 2020-11-16T22:18:42.058 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:18:42.059 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.041 ******* 2020-11-16T22:18:42.098 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:18:42.099 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.131 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.080 ******* 2020-11-16T22:18:42.136 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:18:42.138 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.170 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.119 ******* 2020-11-16T22:18:42.177 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:18:42.179 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.210 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.159 ******* 2020-11-16T22:18:42.216 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:42.218 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi138.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003239", "end": "2020-11-16 22:18:42.179398", "rc": 0, "start": "2020-11-16 22:18:42.176159", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:18:42.253 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.043) 0:02:16.203 ******* 2020-11-16T22:18:42.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.029) 0:02:16.232 ******* 2020-11-16T22:18:42.290 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:18:42.291 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.326 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.043) 0:02:16.275 ******* 2020-11-16T22:18:42.356 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.030) 0:02:16.306 ******* 2020-11-16T22:18:42.363 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:42.364 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-16T22:18:42.365 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.040) 0:02:16.346 ******* 2020-11-16T22:18:42.424 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.028) 0:02:16.374 ******* 2020-11-16T22:18:42.458 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.034) 0:02:16.408 ******* 2020-11-16T22:18:42.465 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:18:42.466 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set root disk] ************************************************ ok: [smithi122.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-16T22:18:42.468 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.498 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.040) 0:02:16.448 ******* 2020-11-16T22:18:42.528 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.029) 0:02:16.477 ******* 2020-11-16T22:18:42.558 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.030) 0:02:16.508 ******* 2020-11-16T22:18:42.563 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:18:42.565 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.597 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.038) 0:02:16.546 ******* 2020-11-16T22:18:42.626 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.029) 0:02:16.575 ******* 2020-11-16T22:18:42.633 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:18:42.635 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.667 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.040) 0:02:16.616 ******* 2020-11-16T22:18:42.674 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:42.675 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.707 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.656 ******* 2020-11-16T22:18:42.714 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:18:42.715 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005012", "end": "2020-11-16 22:18:42.681612", "rc": 0, "start": "2020-11-16 22:18:42.676600", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.696 ******* 2020-11-16T22:18:42.775 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.028) 0:02:16.724 ******* 2020-11-16T22:18:42.781 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:42.782 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.783 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi122.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.815 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.764 ******* 2020-11-16T22:18:42.845 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.029) 0:02:16.794 ******* 2020-11-16T22:18:42.852 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:18:42.853 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:18:42.884 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.039) 0:02:16.834 ******* 2020-11-16T22:18:42.915 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.030) 0:02:16.864 ******* 2020-11-16T22:18:42.922 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:18:42.923 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:42.959 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:42 +0000 (0:00:00.044) 0:02:16.908 ******* 2020-11-16T22:18:42.986 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:18:42.988 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.018 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.059) 0:02:16.968 ******* 2020-11-16T22:18:43.045 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:18:43.046 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.063) 0:02:17.031 ******* 2020-11-16T22:18:43.109 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:18:43.111 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.141 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.058) 0:02:17.090 ******* 2020-11-16T22:18:43.148 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:18:43.150 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-16T22:18:43.180 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.039) 0:02:17.130 ******* 2020-11-16T22:18:43.187 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:18:43.189 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.227 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.045) 0:02:17.176 ******* 2020-11-16T22:18:43.253 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T22:18:43.255 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.285 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.058) 0:02:17.234 ******* 2020-11-16T22:18:43.312 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:18:43.314 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.345 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.060) 0:02:17.294 ******* 2020-11-16T22:18:43.372 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:18:43.374 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.405 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.059) 0:02:17.354 ******* 2020-11-16T22:18:43.411 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:43.413 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:18:43.444 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.039) 0:02:17.394 ******* 2020-11-16T22:18:43.451 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:18:43.453 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.485 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.040) 0:02:17.434 ******* 2020-11-16T22:18:43.513 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:18:43.515 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.072) 0:02:17.507 ******* 2020-11-16T22:18:43.565 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:18:43.567 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-16T22:18:43.599 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.041) 0:02:17.548 ******* 2020-11-16T22:18:43.605 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:18:43.607 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.040) 0:02:17.589 ******* 2020-11-16T22:18:43.647 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:18:43.648 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529394688, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180028, 'block_used': 12915080, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180028, "block_size": 4096, "block_total": 240095108, "block_used": 12915080, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529394688, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-16T22:18:43.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-16T22:18:43.680 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.041) 0:02:17.630 ******* 2020-11-16T22:18:43.710 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.029) 0:02:17.660 ******* 2020-11-16T22:18:43.716 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T22:18:43.718 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.749 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.038) 0:02:17.699 ******* 2020-11-16T22:18:43.755 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:18:43.757 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.787 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.037) 0:02:17.736 ******* 2020-11-16T22:18:43.792 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T22:18:43.794 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.825 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.038) 0:02:17.775 ******* 2020-11-16T22:18:43.831 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:18:43.833 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.864 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.038) 0:02:17.813 ******* 2020-11-16T22:18:43.869 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:18:43.870 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:18:43.872 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:18:43.874 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005098", "end": "2020-11-16 22:18:43.867557", "rc": 0, "start": "2020-11-16 22:18:43.862459", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:18:43.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.039) 0:02:17.853 ******* 2020-11-16T22:18:43.934 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.030) 0:02:17.883 ******* 2020-11-16T22:18:43.942 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:18:43.943 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi138.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529394688, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180028, 'block_used': 12915080, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180028, "block_size": 4096, "block_total": 240095108, "block_used": 12915080, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529394688, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:43.975 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:43 +0000 (0:00:00.041) 0:02:17.925 ******* 2020-11-16T22:18:44.012 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.036) 0:02:17.961 ******* 2020-11-16T22:18:44.019 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:18:44.020 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} ok: [smithi173.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:18:44.022 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:18:44.022 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.023 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.023 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.055 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.042) 0:02:18.004 ******* 2020-11-16T22:18:44.087 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.032) 0:02:18.037 ******* 2020-11-16T22:18:44.094 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:18:44.095 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.128 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.040) 0:02:18.077 ******* 2020-11-16T22:18:44.154 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:44.156 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.186 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.058) 0:02:18.136 ******* 2020-11-16T22:18:44.213 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:44.215 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.245 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.059) 0:02:18.195 ******* 2020-11-16T22:18:44.273 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.274 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi173.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003627", "end": "2020-11-16 22:18:44.251806", "rc": 0, "start": "2020-11-16 22:18:44.248179", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:18:44.308 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.063) 0:02:18.258 ******* 2020-11-16T22:18:44.338 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.029) 0:02:18.288 ******* 2020-11-16T22:18:44.344 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T22:18:44.345 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.042) 0:02:18.330 ******* 2020-11-16T22:18:44.387 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:18:44.389 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.420 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.039) 0:02:18.370 ******* 2020-11-16T22:18:44.427 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T22:18:44.428 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.461 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.040) 0:02:18.410 ******* 2020-11-16T22:18:44.468 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:18:44.470 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.040) 0:02:18.451 ******* 2020-11-16T22:18:44.507 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T22:18:44.509 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.541 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.039) 0:02:18.490 ******* 2020-11-16T22:18:44.548 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:18:44.550 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.550 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:18:44.551 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZT5Bx6vOC5ez3SwLvATaMyBIC0kwygmCT'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.440374", "end": "2020-11-16 22:18:44.522022", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZT5Bx6vOC5ez3SwLvATaMyBIC0kwygmCT"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:43.081648", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:18:44.582 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.040) 0:02:18.531 ******* 2020-11-16T22:18:44.589 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:18:44.590 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.621 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.039) 0:02:18.571 ******* 2020-11-16T22:18:44.628 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:18:44.630 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.039) 0:02:18.610 ******* 2020-11-16T22:18:44.705 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.736 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.075) 0:02:18.685 ******* 2020-11-16T22:18:44.763 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:18:44.765 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.795 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.059) 0:02:18.744 ******* 2020-11-16T22:18:44.822 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:18:44.824 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.854 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.059) 0:02:18.804 ******* 2020-11-16T22:18:44.881 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:18:44.912 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:18:44 +0000 (0:00:00.058) 0:02:18.862 ******* 2020-11-16T22:20:30.439 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:44 +0000 (0:00:00.059) 0:02:18.921 ******* TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.057) 0:02:18.979 ******* TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.056) 0:02:19.036 ******* TASK [testnode : Stop collectl] ************************************************ skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.056) 0:02:19.093 ******* TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.056) 0:02:19.149 ******* TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.056) 0:02:19.206 ******* TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.056) 0:02:19.262 ******* TASK [testnode : set_fact] ****************** 2020-11-16T22:20:30.441 INFO:teuthology.task.ansible.out:*********************************** ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} Monday 16 November 2020 22:18:45 +0000 (0:00:00.055) 0:02:19.318 ******* TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi194.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} ok: [smithi194.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi194.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.115) 0:02:19.433 ******* TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} Monday 16 November 2020 22:18:45 +0000 (0:00:00.096) 0:02:19.530 ******* TASK [testnode : Zap all non-root disks] ************************************** 2020-11-16T22:20:30.442 INFO:teuthology.task.ansible.out:* changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy31htcRX2Zkw2MZR6YgWnbH775LE10N98j'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.296751", "end": "2020-11-16 22:18:45.545891", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy31htcRX2Zkw2MZR6YgWnbH775LE10N98j"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:44.249140", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Set root disk] ************************************************ ok: [smithi173.front.sepia.ceph.com 2020-11-16T22:20:30.444 INFO:teuthology.task.ansible.out:] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529402880, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180030, 'block_used': 12915078, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180030, "block_size": 4096, "block_total": 240095108, "block_used": 12915078, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529402880, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} Monday 16 November 2020 22:18:45 +0000 (0:00:00.072) 0:02:19.602 ******* TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi194.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003589", "end": "2020-11-16 22:18:45.641248", "rc": 0, "start": "2020-11-16 22:18:45.637659", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} Monday 16 November 2020 22:18:45 +0000 (0:00:00.053) 0:02:19.656 ******* TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.061) 0:02:19.717 ******* TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi194.front.sepia.ceph.com] => {"changed": fa 2020-11-16T22:20:30.446 INFO:teuthology.task.ansible.out:lse, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.057) 0:02:19.775 ******* TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-L5t2jv-PkVg-wj5s-NCOq-oMXj-HPgv-jVqQVF', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204010U400BGN', 'nvme-nvme.8086-50484654363230343031305534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.068141", "end": "2020-11-16 22:18:45.792811", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-L5t2jv-PkVg-wj5s-NCOq-oMXj-HPgv-jVqQVF", "nvme-INTEL_SSDPEDMD400G4_PHFT6204010U400BGN", "nvme-nvme.8086-50484654363230343031305534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "37 2020-11-16T22:20:30.447 INFO:teuthology.task.ansible.out:2.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:44.724670", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004959", "end": "2020-11-16 22:18:45.839528", "rc": 0, "start": "2020-11-16 22:18:45.834569", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Monday 16 November 2020 22:18:45 +0000 (0:00:00.050) 0:02:19.825 ******* TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.038) 0:02:19.864 ******* TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi173.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529402880, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180030, 'block_used': 12915078, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180030, "block_size": 4096, "block_total": 240095108, "block_used": 12915078, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": 2020-11-16T22:20:30.449 INFO:teuthology.task.ansible.out: "/", "options": "rw,seclabel,relatime", "size_available": 930529402880, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.039) 0:02:19.903 ******* TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:18:45 +0000 (0:00:00.037) 0:02:19.941 ******* TASK [testnode : Wipe filesystems on non-root partitions] ********************** Monday 16 November 2020 22:18:46 +0000 (0:00:00.039) 0:02:19.981 ******* TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-X5CpLv-gwyf-7S3S-LfOx-61CL-2tD4-lXhEFe', 'nvme-INTEL_SSDPEDMD400G4_CVFT6161003C400BGN', 'nvme-nvme.8086-43564654363136313030334334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.061970", "end": "2020-11-16 22:18:46.802451", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile 2020-11-16T22:20:30.450 INFO:teuthology.task.ansible.out:memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-X5CpLv-gwyf-7S3S-LfOx-61CL-2tD4-lXhEFe", "nvme-INTEL_SSDPEDMD400G4_CVFT6161003C400BGN", "nvme-nvme.8086-43564654363136313030334334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:45.740481", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} Monday 16 November 2020 22:18:46 +0000 (0:00:00.923) 0:02:20.904 ******* TASK [testnode : Set root disk] ************************************************ ok: [smithi194.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, 2020-11-16T22:20:30.452 INFO:teuthology.task.ansible.out: "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} Monday 16 November 2020 22:18:47 +0000 (0:00:00.078) 0:02:20.983 ******* TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZV3DUwup9feH7MZNssBxl2E2hii9Yqg33'], 'uuids': ['d76364d0-b39b-4a6c-aa5a-a80e65662632'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.048976", "end": "2020-11-16 22:18:47.027477", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZV3DUwup9feH7MZNssBxl2E2hii9Yqg33"], "labels": [], "masters": [], "uuids": ["d76364d0-b39b-4a6c-aa5a-a80e65662632"]}, "model": 2020-11-16T22:20:30.452 INFO:teuthology.task.ansible.out:null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:45.978501", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or a 2020-11-16T22:20:30.454 INFO:teuthology.task.ansible.out:fter you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:30.455 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005090", "end": "2020-11-16 22:18:47.223550", "rc": 0, "start": "2020-11-16 22:18:47.218460", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:30.455 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:30.456 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSmqZRCH95BQtyXguQjFrxP0KLBKeyTvj9'], 'uuids': ['8f6e7351-72e6-48c4-a7da-2eef8e6b41ed'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.242995", "end": "2020-11-16 22:18:47.474662", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSmqZRCH95BQtyXguQjFrxP0KLBKeyTvj9"], "labels": [], "masters": [], "uuids": ["8f6e7351-72e6-48c4-a7da-2eef8e6b41ed"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:46.231667", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot 2020-11-16T22:20:30.457 INFO:teuthology.task.ansible.out: or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3J2DKuyq13blCqdpHAyP4sNDhQWWbvaQC'], 'uuids': ['498723bd-3c1d-4364-987d-71af8cebf7d7'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.065429", "end": "2020-11-16 22:18:48.055261", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3J2DKuyq13blCqdpHAyP4sNDhQWWbvaQC"], "labels": [], "masters": [], "uuids": ["498723bd-3c1d-4364-987d-71af8cebf7d7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:46.989832", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGP 2020-11-16T22:20:30.459 INFO:teuthology.task.ansible.out:T data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZdtkklFQgtKptv1SqZDOU06noCgr5z7tc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.049142", "end": "2020-11-16 22:18:48.279778", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZdtkklFQgtKptv1SqZDOU06noCgr5z7tc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:47.230636", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating 2020-11-16T22:20:30.460 INFO:teuthology.task.ansible.out:new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-aL1ACQ-SkwW-KwHe-VAvo-Y0Wm-4kfD-xxYHuh', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN', 'nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.070605", "end": "2020-11-16 22:18:48.734740", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-aL1ACQ-SkwW-KwHe-VAvo-Y0Wm-4kfD-xxYHuh", "nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN", "nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "non 2020-11-16T22:20:30.461 INFO:teuthology.task.ansible.out:e", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:47.664135", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3zwliD1ELLKBf2EcC2ju3xztv7zpJuBE5'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.062444", "end": "2020-11-16 22:18:49.306018", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3zwliD1ELLKBf2EcC2ju3xztv7zpJuBE5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:48.243574", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! Y 2020-11-16T22:20:30.463 INFO:teuthology.task.ansible.out:ou may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZ8zf98IEuN6RZEzoM3bn70S3fWXDghat9'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.046865", "end": "2020-11-16 22:18:49.519688", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZ8zf98IEuN6RZEzoM3bn70S3fWXDghat9"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:48.472823", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: T 2020-11-16T22:20:30.464 INFO:teuthology.task.ansible.out:he kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi122.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L5J2', 'wwn-0x5000c500918e77a4'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5L5J2', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500918e77a4', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L5J2-part1', 'wwn-0x5000c500918e77a4-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L5J2", "wwn-0x5000c500918e77a4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L5J2-part1", "wwn-0x5000c500918e77a4-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_a 2020-11-16T22:20:30.466 INFO:teuthology.task.ansible.out:ddress": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L5J2", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918e77a4"}}, "skip_reason": "Conditional result was False"} changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSI71TN2CQcVRM4aU9kBlRTIocE2mcLRZz'], 'uuids': ['b42e2f5d-078f-48c8-a9a4-1c0dfaec0d3a'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.058698", "end": "2020-11-16 22:18:49.983580", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSI71TN2CQcVRM4aU9kBlRTIocE2mcLRZz"], "labels": [], "masters": [], "uuids": ["b42e2f5d-078f-48c8-a9a4-1c0dfaec0d3a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:48.924882", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: 2020-11-16T22:20:30.467 INFO:teuthology.task.ansible.out:The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3LuPeWOaQe9HGCHEGk7GW1ZgkOeD2sBTL'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.051836", "end": "2020-11-16 22:18:50.547480", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3LuPeWOaQe9HGCHEGk7GW1ZgkOeD2sBTL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:49.495644", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run pa 2020-11-16T22:20:30.469 INFO:teuthology.task.ansible.out:rtprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi138.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKK0', 'wwn-0x5000c5009293769f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XKK0', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c5009293769f', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKK0-part1', 'wwn-0x5000c5009293769f-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKK0", "wwn-0x5000c5009293769f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKK0-part1", "wwn-0x5000c5009293769f-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", 2020-11-16T22:20:30.470 INFO:teuthology.task.ansible.out:"serial": "Z1W5XKK0", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293769f"}}, "skip_reason": "Conditional result was False"} changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZvF4DEKKQ80Kmngvu54GDgfBAowkQaRnw'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.044387", "end": "2020-11-16 22:18:50.762051", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZvF4DEKKQ80Kmngvu54GDgfBAowkQaRnw"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:49.717664", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now par 2020-11-16T22:20:30.472 INFO:teuthology.task.ansible.out:tition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:30.475 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSiadifvlrT1qm0zkC5qbL1HsXuxbSN0Uv'], 'uuids': ['c0cde1c2-218f-4afb-8de8-1a8533f660aa'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.056842", "end": "2020-11-16 22:18:51.234177", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSiadifvlrT1qm0zkC5qbL1HsXuxbSN0Uv"], "labels": [], "masters": [], "uuids": ["c0cde1c2-218f-4afb-8de8-1a8533f660aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:50.177335", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:30.476 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy35ihvjfvjXcvbaZYiuSceqE3L1xsEAp26'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.079502", "end": "2020-11-16 22:18:51.820651", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy35ihvjfvjXcvbaZYiuSceqE3L1xsEAp26"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:50.741149", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:30.482 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSl6JSMfENvkOFw8WSEqoVzgvQtqf7u1C6'], 'uuids': ['4837f6ab-8ecc-410f-8201-51ed3dac1f29'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.050802", "end": "2020-11-16 22:18:52.476025", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSl6JSMfENvkOFw8WSEqoVzgvQtqf7u1C6"], "labels": [], "masters": [], "uuids": ["4837f6ab-8ecc-410f-8201-51ed3dac1f29"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:51.425223", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:30.484 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y', 'wwn-0x5000c50092935686'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XM1Y', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50092935686', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1', 'wwn-0x5000c50092935686-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y", "wwn-0x5000c50092935686"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1", "wwn-0x5000c50092935686-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XM1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092935686"}}, 2020-11-16T22:20:30.485 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSdoY5C2qjhQQXHv9qDJCaqGlXPcdmyxXO'], 'uuids': ['f4a482a2-8d0d-4eae-8d9d-e7c3b14aaa02'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.072196", "end": "2020-11-16 22:18:53.742301", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSdoY5C2qjhQQXHv9qDJCaqGlXPcdmyxXO"], "labels": [], "masters": [], "uuids": ["f4a482a2-8d0d-4eae-8d9d-e7c3b14aaa02"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:18:52.670105", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."] 2020-11-16T22:20:30.487 INFO:teuthology.task.ansible.out:} 2020-11-16T22:20:30.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:01:43.489) 0:04:04.472 ******* 2020-11-16T22:20:30.553 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:00:00.030) 0:04:04.503 ******* 2020-11-16T22:20:30.583 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:00:00.030) 0:04:04.533 ******* 2020-11-16T22:20:30.615 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:00:00.031) 0:04:04.564 ******* 2020-11-16T22:20:30.654 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi194.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:30.687 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:00:00.072) 0:04:04.636 ******* 2020-11-16T22:20:30.713 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:20:30.745 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:30 +0000 (0:00:00.057) 0:04:04.694 ******* 2020-11-16T22:20:31.041 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:20:31.043 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZT5Bx6vOC5ez3SwLvATaMyBIC0kwygmCT'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.135838", "end": "2020-11-16 22:20:31.029123", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZT5Bx6vOC5ez3SwLvATaMyBIC0kwygmCT"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:30.893285", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.133175 s, 866 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.133175 s, 866 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.075 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy31htcRX2Zkw2MZR6YgWnbH775LE10N98j'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.141568", "end": "2020-11-16 22:20:31.063329", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy31htcRX2Zkw2MZR6YgWnbH775LE10N98j"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:30.921761", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.138801 s, 831 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.138801 s, 831 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.107 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSmqZRCH95BQtyXguQjFrxP0KLBKeyTvj9'], 'uuids': ['8f6e7351-72e6-48c4-a7da-2eef8e6b41ed'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.137982", "end": "2020-11-16 22:20:31.097862", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSmqZRCH95BQtyXguQjFrxP0KLBKeyTvj9"], "labels": [], "masters": [], "uuids": ["8f6e7351-72e6-48c4-a7da-2eef8e6b41ed"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:30.959880", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.135374 s, 852 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.135374 s, 852 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.255 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-L5t2jv-PkVg-wj5s-NCOq-oMXj-HPgv-jVqQVF', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204010U400BGN', 'nvme-nvme.8086-50484654363230343031305534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036794", "end": "2020-11-16 22:20:31.246943", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-L5t2jv-PkVg-wj5s-NCOq-oMXj-HPgv-jVqQVF", "nvme-INTEL_SSDPEDMD400G4_PHFT6204010U400BGN", "nvme-nvme.8086-50484654363230343031305534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.210149", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0343582 s, 3.4 GB/s", "std 2020-11-16T22:20:31.257 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0343582 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.298 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-X5CpLv-gwyf-7S3S-LfOx-61CL-2tD4-lXhEFe', 'nvme-INTEL_SSDPEDMD400G4_CVFT6161003C400BGN', 'nvme-nvme.8086-43564654363136313030334334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037307", "end": "2020-11-16 22:20:31.283621", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-X5CpLv-gwyf-7S3S-LfOx-61CL-2tD4-lXhEFe", "nvme-INTEL_SSDPEDMD400G4_CVFT6161003C400BGN", "nvme-nvme.8086-43564654363136313030334334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.246314", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0346892 s, 3.3 GB/s", "std 2020-11-16T22:20:31.300 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0346892 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.331 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-aL1ACQ-SkwW-KwHe-VAvo-Y0Wm-4kfD-xxYHuh', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN', 'nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037111", "end": "2020-11-16 22:20:31.315761", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-aL1ACQ-SkwW-KwHe-VAvo-Y0Wm-4kfD-xxYHuh", "nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN", "nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.278650", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0345767 s, 3.3 GB/s", "std 2020-11-16T22:20:31.333 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0345767 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.581 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZV3DUwup9feH7MZNssBxl2E2hii9Yqg33'], 'uuids': ['d76364d0-b39b-4a6c-aa5a-a80e65662632'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.135970", "end": "2020-11-16 22:20:31.569573", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZV3DUwup9feH7MZNssBxl2E2hii9Yqg33"], "labels": [], "masters": [], "uuids": ["d76364d0-b39b-4a6c-aa5a-a80e65662632"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.433603", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.133408 s, 865 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.133408 s, 865 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.623 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3J2DKuyq13blCqdpHAyP4sNDhQWWbvaQC'], 'uuids': ['498723bd-3c1d-4364-987d-71af8cebf7d7'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.139267", "end": "2020-11-16 22:20:31.613838", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3J2DKuyq13blCqdpHAyP4sNDhQWWbvaQC"], "labels": [], "masters": [], "uuids": ["498723bd-3c1d-4364-987d-71af8cebf7d7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.474571", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136718 s, 844 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136718 s, 844 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.656 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSI71TN2CQcVRM4aU9kBlRTIocE2mcLRZz'], 'uuids': ['b42e2f5d-078f-48c8-a9a4-1c0dfaec0d3a'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.139218", "end": "2020-11-16 22:20:31.647399", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSI71TN2CQcVRM4aU9kBlRTIocE2mcLRZz"], "labels": [], "masters": [], "uuids": ["b42e2f5d-078f-48c8-a9a4-1c0dfaec0d3a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.508181", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136573 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136573 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.904 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZdtkklFQgtKptv1SqZDOU06noCgr5z7tc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.138428", "end": "2020-11-16 22:20:31.889262", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZdtkklFQgtKptv1SqZDOU06noCgr5z7tc"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.750834", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.135797 s, 849 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.135797 s, 849 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.937 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3zwliD1ELLKBf2EcC2ju3xztv7zpJuBE5'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.137706", "end": "2020-11-16 22:20:31.927824", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3zwliD1ELLKBf2EcC2ju3xztv7zpJuBE5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.790118", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.135238 s, 853 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.135238 s, 853 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:31.980 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSiadifvlrT1qm0zkC5qbL1HsXuxbSN0Uv'], 'uuids': ['c0cde1c2-218f-4afb-8de8-1a8533f660aa'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.139913", "end": "2020-11-16 22:20:31.964150", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSiadifvlrT1qm0zkC5qbL1HsXuxbSN0Uv"], "labels": [], "masters": [], "uuids": ["c0cde1c2-218f-4afb-8de8-1a8533f660aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.824237", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137372 s, 840 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137372 s, 840 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.220 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZ8zf98IEuN6RZEzoM3bn70S3fWXDghat9'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.140460", "end": "2020-11-16 22:20:32.209804", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZ8zf98IEuN6RZEzoM3bn70S3fWXDghat9"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.069344", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137899 s, 836 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137899 s, 836 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.232 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L5J2', 'wwn-0x5000c500918e77a4'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5L5J2', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500918e77a4', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L5J2-part1', 'wwn-0x5000c500918e77a4-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L5J2", "wwn-0x5000c500918e77a4"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L5J2-part1", "wwn-0x5000c500918e77a4-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L5J2", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918e77a4"}}, 2020-11-16T22:20:32.234 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3M8ZfAJwPlheKJffxP3FsiA0dQsyw0qCY'], 'uuids': ['fcc24499-78df-4ade-8bf3-0ff2ba2396ef'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.099540", "end": "2020-11-16 22:20:32.219039", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3M8ZfAJwPlheKJffxP3FsiA0dQsyw0qCY"], "labels": [], "masters": [], "uuids": ["fcc24499-78df-4ade-8bf3-0ff2ba2396ef"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:31.119499", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data struc 2020-11-16T22:20:32.235 INFO:teuthology.task.ansible.out:tures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:32.265 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:20:32.267 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3LuPeWOaQe9HGCHEGk7GW1ZgkOeD2sBTL'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.152416", "end": "2020-11-16 22:20:32.259405", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy3LuPeWOaQe9HGCHEGk7GW1ZgkOeD2sBTL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.106989", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149893 s, 770 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149893 s, 770 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.277 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKK0', 'wwn-0x5000c5009293769f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XKK0', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c5009293769f', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKK0-part1', 'wwn-0x5000c5009293769f-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKK0", "wwn-0x5000c5009293769f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKK0-part1", "wwn-0x5000c5009293769f-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XKK0", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293769f"}}, 2020-11-16T22:20:32.279 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.299 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSl6JSMfENvkOFw8WSEqoVzgvQtqf7u1C6'], 'uuids': ['4837f6ab-8ecc-410f-8201-51ed3dac1f29'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.141859", "end": "2020-11-16 22:20:32.284650", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSl6JSMfENvkOFw8WSEqoVzgvQtqf7u1C6"], "labels": [], "masters": [], "uuids": ["4837f6ab-8ecc-410f-8201-51ed3dac1f29"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.142791", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139264 s, 828 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139264 s, 828 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.300 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y', 'wwn-0x5000c50092935686'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XM1Y', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50092935686', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1', 'wwn-0x5000c50092935686-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y", "wwn-0x5000c50092935686"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1", "wwn-0x5000c50092935686-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XM1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092935686"}}, 2020-11-16T22:20:32.302 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:20:32.540 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZvF4DEKKQ80Kmngvu54GDgfBAowkQaRnw'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.134842", "end": "2020-11-16 22:20:32.529756", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-94lO7H1epyPO2PYLqnE2BaAnmcgchEzZvF4DEKKQ80Kmngvu54GDgfBAowkQaRnw"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.394914", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132438 s, 871 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132438 s, 871 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.577 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:01.832) 0:04:06.527 ******* 2020-11-16T22:20:32.595 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy35ihvjfvjXcvbaZYiuSceqE3L1xsEAp26'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.136128", "end": "2020-11-16 22:20:32.582100", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-D6uag4JSDRdWKA8hoMC8YHGoyXCBaZy35ihvjfvjXcvbaZYiuSceqE3L1xsEAp26"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.445972", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.133584 s, 863 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.133584 s, 863 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.634 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.057) 0:04:06.584 ******* 2020-11-16T22:20:32.642 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSdoY5C2qjhQQXHv9qDJCaqGlXPcdmyxXO'], 'uuids': ['f4a482a2-8d0d-4eae-8d9d-e7c3b14aaa02'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.149962", "end": "2020-11-16 22:20:32.617540", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eWuIdrMkszZIwy3nWEhHhkPNKU1sLsFSdoY5C2qjhQQXHv9qDJCaqGlXPcdmyxXO"], "labels": [], "masters": [], "uuids": ["f4a482a2-8d0d-4eae-8d9d-e7c3b14aaa02"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.467578", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147432 s, 782 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147432 s, 782 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.681 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.046) 0:04:06.630 ******* 2020-11-16T22:20:32.843 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T22:20:32.845 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.068628", "end": "2020-11-16 22:20:32.827189", "rc": 0, "start": "2020-11-16 22:20:32.758561", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.877 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.195) 0:04:06.826 ******* 2020-11-16T22:20:32.894 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.062020", "end": "2020-11-16 22:20:32.876158", "rc": 0, "start": "2020-11-16 22:20:32.814138", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.927 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.050) 0:04:06.876 ******* 2020-11-16T22:20:32.945 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.073815", "end": "2020-11-16 22:20:32.935690", "rc": 0, "start": "2020-11-16 22:20:32.861875", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:32.978 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:32 +0000 (0:00:00.051) 0:04:06.928 ******* 2020-11-16T22:20:33.109 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T22:20:33.111 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049380", "end": "2020-11-16 22:20:33.106148", "rc": 0, "start": "2020-11-16 22:20:33.056768", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:33.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.164) 0:04:07.092 ******* 2020-11-16T22:20:33.170 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:20:33.172 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.051825", "end": "2020-11-16 22:20:33.160870", "rc": 0, "start": "2020-11-16 22:20:33.109045", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:33.204 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.061) 0:04:07.154 ******* 2020-11-16T22:20:33.234 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.029) 0:04:07.184 ******* 2020-11-16T22:20:33.241 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.061933", "end": "2020-11-16 22:20:33.220719", "rc": 0, "start": "2020-11-16 22:20:33.158786", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:33.241 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:20:33.242 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.275 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.040) 0:04:07.225 ******* 2020-11-16T22:20:33.305 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.029) 0:04:07.254 ******* 2020-11-16T22:20:33.310 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:20:33.312 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.342 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.037) 0:04:07.292 ******* 2020-11-16T22:20:33.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.029) 0:04:07.322 ******* 2020-11-16T22:20:33.378 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:20:33.380 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.412 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.039) 0:04:07.362 ******* 2020-11-16T22:20:33.446 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.033) 0:04:07.395 ******* 2020-11-16T22:20:33.452 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:20:33.454 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.484 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.038) 0:04:07.434 ******* 2020-11-16T22:20:33.518 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.033) 0:04:07.467 ******* 2020-11-16T22:20:33.525 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:20:33.526 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:33.527 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-cBFmJh-jsCb-9dlf-6UpC-fzsm-nTa4-2XnRr8', 'nvme-INTEL_SSDPEDMD400G4_CVFT6240002A400BGN', 'nvme-nvme.8086-43564654363234303030324134303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.059771", "end": "2020-11-16 22:20:33.470906", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-cBFmJh-jsCb-9dlf-6UpC-fzsm-nTa4-2XnRr8", "nvme-INTEL_SSDPEDMD400G4_CVFT6240002A400BGN", "nvme-nvme.8086-43564654363234303030324134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_dis 2020-11-16T22:20:33.528 INFO:teuthology.task.ansible.out:card": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:32.411135", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:33.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.044) 0:04:07.512 ******* 2020-11-16T22:20:33.569 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:20:33.570 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.600 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.038) 0:04:07.550 ******* 2020-11-16T22:20:33.607 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.038) 0:04:07.588 ******* 2020-11-16T22:20:33.644 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:20:33.646 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.678 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.039) 0:04:07.627 ******* 2020-11-16T22:20:33.685 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.685 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:20:33.686 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:20:33.717 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.039) 0:04:07.667 ******* 2020-11-16T22:20:33.749 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.031) 0:04:07.698 ******* 2020-11-16T22:20:33.754 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:20:33.756 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set root disk] ************************************************ skipping: [smithi122.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.787 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.038) 0:04:07.737 ******* 2020-11-16T22:20:33.816 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.028) 0:04:07.766 ******* 2020-11-16T22:20:33.823 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:20:33.824 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.858 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.041) 0:04:07.808 ******* 2020-11-16T22:20:33.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.030) 0:04:07.838 ******* 2020-11-16T22:20:33.896 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:33.898 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:33.931 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.041) 0:04:07.880 ******* 2020-11-16T22:20:33.960 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:33.962 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.000 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:33 +0000 (0:00:00.069) 0:04:07.950 ******* 2020-11-16T22:20:34.020 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:20:34.021 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:20:34.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.051) 0:04:08.002 ******* 2020-11-16T22:20:34.059 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:20:34.061 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.094 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.043 ******* 2020-11-16T22:20:34.101 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:20:34.102 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:20:34.104 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:20:34.104 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529394688, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180028, 'block_used': 12915080, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180028, "block_size": 4096, "block_total": 240095108, "block_used": 12915080, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529394688, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.139 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.045) 0:04:08.088 ******* 2020-11-16T22:20:34.187 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.048) 0:04:08.137 ******* 2020-11-16T22:20:34.194 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:20:34.196 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.228 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.040) 0:04:08.177 ******* 2020-11-16T22:20:34.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.030) 0:04:08.208 ******* 2020-11-16T22:20:34.266 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:20:34.267 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529402880, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180030, 'block_used': 12915078, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180030, "block_size": 4096, "block_total": 240095108, "block_used": 12915078, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529402880, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.268 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.302 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.043) 0:04:08.252 ******* 2020-11-16T22:20:34.334 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.031) 0:04:08.284 ******* 2020-11-16T22:20:34.342 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:34.343 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.375 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.040) 0:04:08.324 ******* 2020-11-16T22:20:34.382 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:20:34.384 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.433 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.057) 0:04:08.382 ******* 2020-11-16T22:20:34.443 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:34.445 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.477 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.044) 0:04:08.426 ******* 2020-11-16T22:20:34.484 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:34.486 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.518 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.468 ******* 2020-11-16T22:20:34.526 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:20:34.528 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.040) 0:04:08.508 ******* 2020-11-16T22:20:34.566 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:34.568 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.600 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.550 ******* 2020-11-16T22:20:34.608 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:20:34.610 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.642 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.591 ******* 2020-11-16T22:20:34.648 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:20:34.650 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.682 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.040) 0:04:08.632 ******* 2020-11-16T22:20:34.688 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:20:34.690 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.724 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.673 ******* 2020-11-16T22:20:34.731 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:20:34.733 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.765 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.041) 0:04:08.715 ******* 2020-11-16T22:20:34.772 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:34.774 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3D9nllZ3DA6f9MREfXXRLdu0VdjILHcff'], 'uuids': ['426caba1-c204-44bd-8d39-9b96b54902f8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.050991", "end": "2020-11-16 22:20:34.717523", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3D9nllZ3DA6f9MREfXXRLdu0VdjILHcff"], "labels": [], "masters": [], "uuids": ["426caba1-c204-44bd-8d39-9b96b54902f8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:33.666532", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:34.795 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:20:34.797 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:34.827 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:34 +0000 (0:00:00.062) 0:04:08.777 ******* 2020-11-16T22:20:35.024 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:20:35.057 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:35 +0000 (0:00:00.229) 0:04:09.007 ******* 2020-11-16T22:20:35.158 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:20:35.190 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:35 +0000 (0:00:00.133) 0:04:09.140 ******* 2020-11-16T22:20:35.291 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:20:35.328 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:35 +0000 (0:00:00.137) 0:04:09.277 ******* 2020-11-16T22:20:35.658 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:20:35.660 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:35.680 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:35.691 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:35.983 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:35.985 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3OYLJFaxvM5e1OhB0cJUWpD3jk8HmTUbG'], 'uuids': ['94eab64c-eb43-4d50-b6e2-cf2ef75dc888'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.050485", "end": "2020-11-16 22:20:35.967647", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3OYLJFaxvM5e1OhB0cJUWpD3jk8HmTUbG"], "labels": [], "masters": [], "uuids": ["94eab64c-eb43-4d50-b6e2-cf2ef75dc888"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:34.917162", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:35.985 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] * 2020-11-16T22:20:35.987 INFO:teuthology.task.ansible.out:************************************ changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.008 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.019 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.311 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.333 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.344 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.657 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.668 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.679 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:36.980 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:37.014 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:01.685) 0:04:10.963 ******* 2020-11-16T22:20:37.021 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:37.023 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:37.057 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:00.043) 0:04:11.007 ******* 2020-11-16T22:20:37.088 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:00.030) 0:04:11.037 ******* 2020-11-16T22:20:37.210 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:20:37.211 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:20:37.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:00.155) 0:04:11.192 ******* 2020-11-16T22:20:37.251 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:20:37.253 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:37.253 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3iOB839kwfxujZzu9M4WWI2gkcBSacbA3'], 'uuids': ['50e5a0d3-050b-4dfa-9c21-4449346bc75c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.066781", "end": "2020-11-16 22:20:37.234680", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3iOB839kwfxujZzu9M4WWI2gkcBSacbA3"], "labels": [], "masters": [], "uuids": ["50e5a0d3-050b-4dfa-9c21-4449346bc75c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:36.167899", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:37.255 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XNHZ', 'wwn-0x5000c50092941eb0'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XNHZ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50092941eb0', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XNHZ-part1', 'wwn-0x5000c50092941eb0-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XNHZ", "wwn-0x5000c50092941eb0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XNHZ-part1", "wwn-0x5000c50092941eb0-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XNHZ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092941eb0"}}, 2020-11-16T22:20:37.256 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:20:37.288 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:00.045) 0:04:11.237 ******* 2020-11-16T22:20:37.295 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:20:37.297 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:20:37.329 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:37 +0000 (0:00:00.040) 0:04:11.278 ******* 2020-11-16T22:20:37.450 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:20:37.452 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.495 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.538 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.632 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.675 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.718 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.802 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.845 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.898 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.983 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:37.994 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.033 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.704) 0:04:11.982 ******* 2020-11-16T22:20:38.041 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:38.043 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.078 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.045) 0:04:12.028 ******* 2020-11-16T22:20:38.086 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:38.089 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:38.128 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.049) 0:04:12.077 ******* 2020-11-16T22:20:38.240 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T22:20:38.242 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:38.289 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.161) 0:04:12.239 ******* 2020-11-16T22:20:38.299 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:38.334 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.044) 0:04:12.283 ******* 2020-11-16T22:20:38.341 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:20:38.343 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:38.378 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.044) 0:04:12.327 ******* 2020-11-16T22:20:38.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.031) 0:04:12.359 ******* 2020-11-16T22:20:38.418 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:20:38.420 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:20:38.452 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.042) 0:04:12.401 ******* 2020-11-16T22:20:38.460 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:20:38.496 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.044) 0:04:12.446 ******* 2020-11-16T22:20:38.505 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:20:38.506 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3ORz5Qq1aU7kmddH0sxyQba2WcDYDy90q'], 'uuids': ['2dee567f-297d-4f40-8343-96f4638669e2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.053664", "end": "2020-11-16 22:20:38.483321", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3ORz5Qq1aU7kmddH0sxyQba2WcDYDy90q"], "labels": [], "masters": [], "uuids": ["2dee567f-297d-4f40-8343-96f4638669e2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:37.429657", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:20:38.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:38 +0000 (0:00:00.049) 0:04:12.495 ******* 2020-11-16T22:20:38.908 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:20:38.910 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3M8ZfAJwPlheKJffxP3FsiA0dQsyw0qCY'], 'uuids': ['fcc24499-78df-4ade-8bf3-0ff2ba2396ef'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.140910", "end": "2020-11-16 22:20:38.899669", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3M8ZfAJwPlheKJffxP3FsiA0dQsyw0qCY"], "labels": [], "masters": [], "uuids": ["fcc24499-78df-4ade-8bf3-0ff2ba2396ef"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:38.758759", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.138225 s, 834 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.138225 s, 834 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:39.128 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-cBFmJh-jsCb-9dlf-6UpC-fzsm-nTa4-2XnRr8', 'nvme-INTEL_SSDPEDMD400G4_CVFT6240002A400BGN', 'nvme-nvme.8086-43564654363234303030324134303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037258", "end": "2020-11-16 22:20:39.114566", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-cBFmJh-jsCb-9dlf-6UpC-fzsm-nTa4-2XnRr8", "nvme-INTEL_SSDPEDMD400G4_CVFT6240002A400BGN", "nvme-nvme.8086-43564654363234303030324134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:39.077308", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0345982 s, 3.3 GB/s", "std 2020-11-16T22:20:39.130 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0345982 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:39.462 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3D9nllZ3DA6f9MREfXXRLdu0VdjILHcff'], 'uuids': ['426caba1-c204-44bd-8d39-9b96b54902f8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.141845", "end": "2020-11-16 22:20:39.447984", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3D9nllZ3DA6f9MREfXXRLdu0VdjILHcff"], "labels": [], "masters": [], "uuids": ["426caba1-c204-44bd-8d39-9b96b54902f8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:39.306139", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139239 s, 828 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139239 s, 828 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:39.783 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3OYLJFaxvM5e1OhB0cJUWpD3jk8HmTUbG'], 'uuids': ['94eab64c-eb43-4d50-b6e2-cf2ef75dc888'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.139156", "end": "2020-11-16 22:20:39.772117", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3OYLJFaxvM5e1OhB0cJUWpD3jk8HmTUbG"], "labels": [], "masters": [], "uuids": ["94eab64c-eb43-4d50-b6e2-cf2ef75dc888"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:39.632961", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136556 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136556 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:39.918 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:20:39.920 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:20:39.953 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:01.407) 0:04:13.903 ******* 2020-11-16T22:20:39.960 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:20:39.961 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:20:39.996 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:39 +0000 (0:00:00.043) 0:04:13.946 ******* 2020-11-16T22:20:40.027 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:04:13.976 ******* 2020-11-16T22:20:40.108 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:20:40.110 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3iOB839kwfxujZzu9M4WWI2gkcBSacbA3'], 'uuids': ['50e5a0d3-050b-4dfa-9c21-4449346bc75c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.148303", "end": "2020-11-16 22:20:40.097815", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3iOB839kwfxujZzu9M4WWI2gkcBSacbA3"], "labels": [], "masters": [], "uuids": ["50e5a0d3-050b-4dfa-9c21-4449346bc75c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:39.949512", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.145806 s, 791 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.145806 s, 791 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:40.121 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XNHZ', 'wwn-0x5000c50092941eb0'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XNHZ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50092941eb0', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XNHZ-part1', 'wwn-0x5000c50092941eb0-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XNHZ", "wwn-0x5000c50092941eb0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XNHZ-part1", "wwn-0x5000c50092941eb0-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XNHZ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092941eb0"}}, 2020-11-16T22:20:40.122 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.445 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3ORz5Qq1aU7kmddH0sxyQba2WcDYDy90q'], 'uuids': ['2dee567f-297d-4f40-8343-96f4638669e2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.146065", "end": "2020-11-16 22:20:40.432411", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TIq3AxKdsFonId67ovmBW62oFwAGQGx3ORz5Qq1aU7kmddH0sxyQba2WcDYDy90q"], "labels": [], "masters": [], "uuids": ["2dee567f-297d-4f40-8343-96f4638669e2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:20:40.286346", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.143644 s, 803 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.143644 s, 803 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:40.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.456) 0:04:14.433 ******* 2020-11-16T22:20:40.512 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:20:40.514 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:20:40.546 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.062) 0:04:14.496 ******* 2020-11-16T22:20:40.554 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:20:40.557 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:20:40.589 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.042) 0:04:14.538 ******* 2020-11-16T22:20:40.621 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.032) 0:04:14.570 ******* 2020-11-16T22:20:40.628 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:20:40.630 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.661 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.040) 0:04:14.611 ******* 2020-11-16T22:20:40.694 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.032) 0:04:14.643 ******* 2020-11-16T22:20:40.701 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.702 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:20:40.702 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.734 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.040) 0:04:14.684 ******* 2020-11-16T22:20:40.765 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:04:14.715 ******* 2020-11-16T22:20:40.773 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.774 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi194.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.081724", "end": "2020-11-16 22:20:40.745273", "rc": 0, "start": "2020-11-16 22:20:40.663549", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:40.808 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.042) 0:04:14.757 ******* 2020-11-16T22:20:40.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.030) 0:04:14.788 ******* 2020-11-16T22:20:40.870 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.031) 0:04:14.820 ******* 2020-11-16T22:20:40.879 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:20:40.880 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:40.914 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.043) 0:04:14.864 ******* 2020-11-16T22:20:40.948 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.033) 0:04:14.897 ******* 2020-11-16T22:20:40.980 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:40 +0000 (0:00:00.032) 0:04:14.930 ******* 2020-11-16T22:20:40.987 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:20:40.989 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.020 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.039) 0:04:14.969 ******* 2020-11-16T22:20:41.050 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:04:14.999 ******* 2020-11-16T22:20:41.056 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:20:41.058 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.095 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.045) 0:04:15.045 ******* 2020-11-16T22:20:41.126 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:04:15.075 ******* 2020-11-16T22:20:41.132 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:20:41.134 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for physical volumes] *********************************** changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.043635", "end": "2020-11-16 22:20:41.095193", "rc": 0, "start": "2020-11-16 22:20:41.051558", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:41.136 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.168 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.041) 0:04:15.117 ******* 2020-11-16T22:20:41.199 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:04:15.148 ******* 2020-11-16T22:20:41.230 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.179 ******* 2020-11-16T22:20:41.236 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:20:41.238 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.269 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.039) 0:04:15.219 ******* 2020-11-16T22:20:41.300 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:04:15.249 ******* 2020-11-16T22:20:41.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.281 ******* 2020-11-16T22:20:41.340 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:20:41.341 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.041) 0:04:15.322 ******* 2020-11-16T22:20:41.404 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.353 ******* 2020-11-16T22:20:41.436 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.032) 0:04:15.385 ******* 2020-11-16T22:20:41.443 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.445 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.477 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.041) 0:04:15.427 ******* 2020-11-16T22:20:41.509 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.458 ******* 2020-11-16T22:20:41.539 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.030) 0:04:15.489 ******* 2020-11-16T22:20:41.548 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:20:41.550 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the ntp.conf file.] ************************************ skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.041) 0:04:15.530 ******* 2020-11-16T22:20:41.613 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.562 ******* 2020-11-16T22:20:41.644 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.031) 0:04:15.593 ******* 2020-11-16T22:20:41.649 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:20:41.651 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.682 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.038) 0:04:15.632 ******* 2020-11-16T22:20:41.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.028) 0:04:15.661 ******* 2020-11-16T22:20:41.717 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:20:41.719 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.755 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.043) 0:04:15.704 ******* 2020-11-16T22:20:41.783 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.028) 0:04:15.733 ******* 2020-11-16T22:20:41.789 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:20:41.791 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the ntp.conf file.] ************************************ skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.823 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.039) 0:04:15.772 ******* 2020-11-16T22:20:41.852 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.029) 0:04:15.801 ******* 2020-11-16T22:20:41.859 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:20:41.860 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.895 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.043) 0:04:15.845 ******* 2020-11-16T22:20:41.902 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:20:41.904 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.936 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.040) 0:04:15.885 ******* 2020-11-16T22:20:41.943 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:20:41.944 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:41.976 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:41 +0000 (0:00:00.039) 0:04:15.925 ******* 2020-11-16T22:20:41.982 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T22:20:41.984 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.017 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.041) 0:04:15.966 ******* 2020-11-16T22:20:42.035 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:20:42.036 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-16T22:20:42.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.050) 0:04:16.017 ******* 2020-11-16T22:20:42.076 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:20:42.077 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.109 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.041) 0:04:16.059 ******* 2020-11-16T22:20:42.179 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:20:42.181 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:20:42.212 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.102) 0:04:16.161 ******* 2020-11-16T22:20:42.249 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:20:42.250 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529398784, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227180029, 'block_used': 12915079, 'inode_total': 61054976, 'inode_available': 60990125, 'inode_used': 64851, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227180029, "block_size": 4096, "block_total": 240095108, "block_used": 12915079, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990125, "inode_total": 61054976, "inode_used": 64851, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529398784, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.282 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.070) 0:04:16.232 ******* 2020-11-16T22:20:42.288 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:20:42.290 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-16T22:20:42.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.272 ******* 2020-11-16T22:20:42.329 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:20:42.330 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.312 ******* 2020-11-16T22:20:42.380 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:20:42.381 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:31 UTC", "ActiveEnterTimestampMonotonic": "15393649", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket systemd-tmpfiles-setup.service sntp.service ntpdate.service ntpd.service basic.target -.mount system.slice sysinit.target tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:30 UTC", "AssertTimestampMonotonic": "14388551", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:30 UTC", "ConditionTimestampMonotonic": "14388521", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-16T22:20:42.383 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2436", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:31 UTC", "ExecMainStartTimestampMonotonic": "15304126", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:31 UTC", "InactiveExitTimestampMonotonic": "14540726", "InvocationID": "b378665ae56f4476bf0064421569ef19", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-16T22:20:42.384 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2436", "MemoryAccounting": "yes", "MemoryCurrent": "2551808", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-16T22:20:42.386 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:31 UTC", "StateChangeTimestampMonotonic": "15393649", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:31 UTC", "WatchdogTimestampMonotonic": "15304140", "WatchdogUSec": "0"}} 2020-11-16T22:20:42.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.051) 0:04:16.364 ******* 2020-11-16T22:20:42.422 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:42.423 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.455 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.404 ******* 2020-11-16T22:20:42.461 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:42.463 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:20:42.495 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.445 ******* 2020-11-16T22:20:42.503 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:42.505 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.537 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.486 ******* 2020-11-16T22:20:42.544 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:42.545 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-16T22:20:42.577 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.040) 0:04:16.527 ******* 2020-11-16T22:20:42.585 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:20:42.587 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the chrony.conf file] ********************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-16T22:20:42.626 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.048) 0:04:16.576 ******* 2020-11-16T22:20:42.778 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.151) 0:04:16.727 ******* 2020-11-16T22:20:42.785 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:20:42.786 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:20:42.788 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:37 UTC", "ActiveEnterTimestampMonotonic": "15497941", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target ntpdate.service system.slice ntpd.service sntp.service -.mount sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:36 UTC", "AssertTimestampMonotonic": "14299197", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:36 UTC", "ConditionTimestampMonotonic": "14299168", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-16T22:20:42.789 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2451", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:37 UTC", "ExecMainStartTimestampMonotonic": "15415576", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:36 UTC", "InactiveExitTimestampMonotonic": "14317098", "InvocationID": "07531f250e104b5cb2aa57a9d3c54692", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-16T22:20:42.791 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2451", "MemoryAccounting": "yes", "MemoryCurrent": "2707456", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-16T22:20:42.792 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:37 UTC", "StateChangeTimestampMonotonic": "15497941", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:37 UTC", "WatchdogTimestampMonotonic": "15415590", "WatchdogUSec": "0"}} 2020-11-16T22:20:42.822 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.044) 0:04:16.771 ******* 2020-11-16T22:20:42.850 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.028) 0:04:16.800 ******* 2020-11-16T22:20:42.857 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:20:42.859 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.038) 0:04:16.839 ******* 2020-11-16T22:20:42.919 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.029) 0:04:16.868 ******* 2020-11-16T22:20:42.925 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:42.927 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:42.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.039) 0:04:16.908 ******* 2020-11-16T22:20:42.988 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:42 +0000 (0:00:00.029) 0:04:16.937 ******* 2020-11-16T22:20:42.995 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:20:42.997 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.029 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:04:16.978 ******* 2020-11-16T22:20:43.035 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:43.037 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-16T22:20:43.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.039) 0:04:17.018 ******* 2020-11-16T22:20:43.076 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:20:43.077 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:29 UTC", "ActiveEnterTimestampMonotonic": "15548304", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target sntp.service system.slice ntpdate.service sysinit.target systemd-tmpfiles-setup.service -.mount ntpd.service systemd-journald.socket tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:28 UTC", "AssertTimestampMonotonic": "14419780", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:28 UTC", "ConditionTimestampMonotonic": "14419751", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service ntpd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-16T22:20:43.078 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2445", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:29 UTC", "ExecMainStartTimestampMonotonic": "15475217", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:28 UTC", "InactiveExitTimestampMonotonic": "14485809", "InvocationID": "f98c8af16afa491ab188c013ddbf742a", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-16T22:20:43.080 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2445", "MemoryAccounting": "yes", "MemoryCurrent": "2633728", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-16T22:20:43.081 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:29 UTC", "StateChangeTimestampMonotonic": "15548304", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:29 UTC", "WatchdogTimestampMonotonic": "15475231", "WatchdogUSec": "0"}} 2020-11-16T22:20:43.110 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.041) 0:04:17.060 ******* 2020-11-16T22:20:43.117 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:20:43.119 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.149 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.038) 0:04:17.098 ******* 2020-11-16T22:20:43.155 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:43.157 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:20:43.188 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.039) 0:04:17.138 ******* 2020-11-16T22:20:43.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.028) 0:04:17.166 ******* 2020-11-16T22:20:43.225 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:20:43.227 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.257 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:04:17.207 ******* 2020-11-16T22:20:43.264 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:43.266 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-16T22:20:43.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.039) 0:04:17.247 ******* 2020-11-16T22:20:43.304 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:20:43.306 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.337 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.039) 0:04:17.286 ******* 2020-11-16T22:20:43.344 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:20:43.345 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.377 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:04:17.326 ******* 2020-11-16T22:20:43.404 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:20:43.406 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.435 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.058) 0:04:17.385 ******* 2020-11-16T22:20:43.478 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:20:43.479 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:43.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.073) 0:04:17.458 ******* 2020-11-16T22:20:43.516 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T22:20:43.517 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:20:43.551 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.042) 0:04:17.501 ******* 2020-11-16T22:20:43.558 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:20:43.559 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:20:43.592 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.040) 0:04:17.541 ******* 2020-11-16T22:20:43.661 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:20:43.663 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:20:43.695 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.102) 0:04:17.644 ******* 2020-11-16T22:20:43.713 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:20:43.714 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:20:43.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:43 +0000 (0:00:00.051) 0:04:17.695 ******* 2020-11-16T22:20:43.941 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:20:43.942 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:44.066 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:20:44.068 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:20:44.098 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.352) 0:04:18.048 ******* 2020-11-16T22:20:44.232 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:20:44.265 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.167) 0:04:18.215 ******* 2020-11-16T22:20:44.273 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:20:44.274 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:44.575 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:44.856 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:20:44.858 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:20:44.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.623) 0:04:18.839 ******* 2020-11-16T22:20:44.908 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:20:44.909 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:44.919 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:20:44.921 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:44.950 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:44 +0000 (0:00:00.061) 0:04:18.900 ******* 2020-11-16T22:20:44.978 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:20:44.979 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:20:45.009 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.058) 0:04:18.959 ******* 2020-11-16T22:20:45.142 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:20:45.144 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:20:45.175 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.165) 0:04:19.124 ******* 2020-11-16T22:20:45.202 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:20:45.204 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.238 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.063) 0:04:19.187 ******* 2020-11-16T22:20:45.268 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.030) 0:04:19.218 ******* 2020-11-16T22:20:45.275 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:20:45.276 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:20:45.278 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.279 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:20:45.310 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.042) 0:04:19.260 ******* 2020-11-16T22:20:45.341 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.030) 0:04:19.291 ******* 2020-11-16T22:20:45.370 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.029) 0:04:19.320 ******* 2020-11-16T22:20:45.377 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:20:45.378 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi122.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.411 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.040) 0:04:19.360 ******* 2020-11-16T22:20:45.441 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.029) 0:04:19.390 ******* 2020-11-16T22:20:45.470 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.029) 0:04:19.419 ******* 2020-11-16T22:20:45.477 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:20:45.478 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.510 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.039) 0:04:19.459 ******* 2020-11-16T22:20:45.518 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:20:45.519 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:20:45.552 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.042) 0:04:19.502 ******* 2020-11-16T22:20:45.571 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:20:45.573 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:20:45.604 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.052) 0:04:19.554 ******* 2020-11-16T22:20:45.622 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:20:45.624 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-16T22:20:45.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.051) 0:04:19.605 ******* 2020-11-16T22:20:45.684 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:20:45.686 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.716 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.059) 0:04:19.665 ******* 2020-11-16T22:20:45.723 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:20:45.724 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:20:45.757 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.040) 0:04:19.706 ******* 2020-11-16T22:20:45.763 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:20:45.765 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi138.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:20:45.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.039) 0:04:19.746 ******* 2020-11-16T22:20:45.827 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.029) 0:04:19.776 ******* 2020-11-16T22:20:45.832 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:20:45.833 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:45.834 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.865 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.038) 0:04:19.815 ******* 2020-11-16T22:20:45.872 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:20:45.873 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:45.905 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.039) 0:04:19.854 ******* 2020-11-16T22:20:45.931 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:20:45.932 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:20:45.962 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:45 +0000 (0:00:00.057) 0:04:19.912 ******* 2020-11-16T22:20:45.970 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:20:45.971 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:20:46.008 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.045) 0:04:19.957 ******* 2020-11-16T22:20:46.016 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:20:46.017 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.048 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.039) 0:04:19.997 ******* 2020-11-16T22:20:46.053 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:20:46.055 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.038) 0:04:20.036 ******* 2020-11-16T22:20:46.092 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:20:46.094 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:20:46.127 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.040) 0:04:20.077 ******* 2020-11-16T22:20:46.134 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:20:46.136 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi122.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.136 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.147 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.181 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.053) 0:04:20.130 ******* 2020-11-16T22:20:46.189 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:20:46.190 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:46.211 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:20:46.213 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.222 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.223 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.234 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.236 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.236 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:20:46.236 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:20:46.267 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.086) 0:04:20.216 ******* 2020-11-16T22:20:46.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.029) 0:04:20.246 ******* 2020-11-16T22:20:46.304 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:46.305 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.339 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.041) 0:04:20.288 ******* 2020-11-16T22:20:46.346 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:20:46.348 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:20:46.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.041) 0:04:20.330 ******* 2020-11-16T22:20:46.386 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.390 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:20:46.391 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.421 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.040) 0:04:20.370 ******* 2020-11-16T22:20:46.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.030) 0:04:20.400 ******* 2020-11-16T22:20:46.457 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:20:46.459 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.493 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.042) 0:04:20.443 ******* 2020-11-16T22:20:46.523 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.029) 0:04:20.472 ******* 2020-11-16T22:20:46.529 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:20:46.531 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.564 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.040) 0:04:20.513 ******* 2020-11-16T22:20:46.594 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.029) 0:04:20.543 ******* 2020-11-16T22:20:46.601 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:20:46.603 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.603 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.604 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.042) 0:04:20.586 ******* 2020-11-16T22:20:46.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.031) 0:04:20.618 ******* 2020-11-16T22:20:46.674 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:20:46.675 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:20:46.677 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.042) 0:04:20.660 ******* 2020-11-16T22:20:46.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.030) 0:04:20.690 ******* 2020-11-16T22:20:46.771 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.029) 0:04:20.720 ******* 2020-11-16T22:20:46.777 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:20:46.779 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.780 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi138.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.780 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.782 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:20:46.783 INFO:teuthology.task.ansible.out:skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi173.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.784 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.784 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.818 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.047) 0:04:20.767 ******* 2020-11-16T22:20:46.846 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.027) 0:04:20.795 ******* 2020-11-16T22:20:46.875 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.029) 0:04:20.825 ******* 2020-11-16T22:20:46.883 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:20:46.884 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi122.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.917 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.041) 0:04:20.866 ******* 2020-11-16T22:20:46.949 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.032) 0:04:20.899 ******* 2020-11-16T22:20:46.979 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:46 +0000 (0:00:00.030) 0:04:20.929 ******* 2020-11-16T22:20:46.986 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:20:46.988 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.989 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:46.991 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.023 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.043) 0:04:20.972 ******* 2020-11-16T22:20:47.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.028) 0:04:21.001 ******* 2020-11-16T22:20:47.057 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:20:47.059 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.091 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.039) 0:04:21.041 ******* 2020-11-16T22:20:47.098 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:47.100 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.132 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.040) 0:04:21.082 ******* 2020-11-16T22:20:47.139 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:20:47.141 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi122.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004808", "end": "2020-11-16 22:20:47.113240", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:20:47.108432", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.171 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.039) 0:04:21.121 ******* 2020-11-16T22:20:47.179 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:20:47.180 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.210 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.038) 0:04:21.159 ******* 2020-11-16T22:20:47.215 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:20:47.217 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.253 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.043) 0:04:21.203 ******* 2020-11-16T22:20:47.259 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:20:47.260 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.292 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.038) 0:04:21.241 ******* 2020-11-16T22:20:47.297 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:47.299 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.038) 0:04:21.280 ******* 2020-11-16T22:20:47.337 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:47.338 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003129", "end": "2020-11-16 22:20:47.289687", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:20:47.286558", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.370 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.039) 0:04:21.320 ******* 2020-11-16T22:20:47.377 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:20:47.378 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.411 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.040) 0:04:21.360 ******* 2020-11-16T22:20:47.417 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:20:47.419 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.040) 0:04:21.401 ******* 2020-11-16T22:20:47.458 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:47.460 INFO:teuthology.task.ansible.out:skipping: [smithi138.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.490 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.038) 0:04:21.440 ******* 2020-11-16T22:20:47.497 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:47.499 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi122.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002688", "end": "2020-11-16 22:20:47.463876", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:20:47.461188", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.532 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.042) 0:04:21.482 ******* 2020-11-16T22:20:47.560 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:20:47.562 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.592 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.060) 0:04:21.542 ******* 2020-11-16T22:20:47.619 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:47.620 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:47.649 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.056) 0:04:21.599 ******* 2020-11-16T22:20:47.687 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:47.688 INFO:teuthology.task.ansible.out:ok: [smithi122.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.038339", "end": "2020-11-16 22:20:47.671229", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:20:47.632890", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.720 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.070) 0:04:21.670 ******* 2020-11-16T22:20:47.728 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004801", "end": "2020-11-16 22:20:47.688326", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:20:47.683525", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.750 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:20:47.781 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:47 +0000 (0:00:00.061) 0:04:21.731 ******* 2020-11-16T22:20:47.861 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:47.862 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004874", "end": "2020-11-16 22:20:47.846880", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:20:47.842006", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.883 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.002836", "end": "2020-11-16 22:20:47.865371", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:20:47.862535", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:47.966 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:20:47.968 INFO:teuthology.task.ansible.out:changed: [smithi122.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:20:48.044 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:48.046 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003001", "end": "2020-11-16 22:20:48.026131", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:20:48.023130", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:48.055 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002668", "end": "2020-11-16 22:20:48.040355", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:20:48.037687", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:48.210 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002647", "end": "2020-11-16 22:20:48.200295", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:20:48.197648", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:48.253 INFO:teuthology.task.ansible.out:ok: [smithi138.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.027588", "end": "2020-11-16 22:20:48.242083", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:20:48.214495", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:48.284 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.503) 0:04:22.234 ******* 2020-11-16T22:20:48.291 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:20:48.292 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:20:48.324 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.039) 0:04:22.273 ******* 2020-11-16T22:20:48.329 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:20:48.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.038) 0:04:22.312 ******* 2020-11-16T22:20:48.412 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:48.414 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.019828", "end": "2020-11-16 22:20:48.398660", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:20:48.378832", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:48.447 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.084) 0:04:22.397 ******* 2020-11-16T22:20:48.474 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:20:48.505 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.057) 0:04:22.454 ******* 2020-11-16T22:20:48.552 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:20:48.554 INFO:teuthology.task.ansible.out:changed: [smithi138.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:20:48.647 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:20:48.678 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.173) 0:04:22.627 ******* 2020-11-16T22:20:48.696 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:20:48.698 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:20:48.727 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:20:48.729 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:48.777 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.098) 0:04:22.726 ******* 2020-11-16T22:20:48.803 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:20:48.805 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:48.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.058) 0:04:22.784 ******* 2020-11-16T22:20:48.862 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:20:48.864 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:48.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.058) 0:04:22.842 ******* 2020-11-16T22:20:48.919 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:20:48.921 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:48.950 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:48 +0000 (0:00:00.057) 0:04:22.900 ******* 2020-11-16T22:20:48.976 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:20:48.977 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.059) 0:04:22.959 ******* 2020-11-16T22:20:49.036 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.066 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.056) 0:04:23.016 ******* 2020-11-16T22:20:49.092 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:20:49.093 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.123 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.056) 0:04:23.073 ******* 2020-11-16T22:20:49.149 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:20:49.150 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.180 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.056) 0:04:23.129 ******* 2020-11-16T22:20:49.206 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:20:49.207 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.239 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.059) 0:04:23.189 ******* 2020-11-16T22:20:49.266 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.301 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.061) 0:04:23.251 ******* 2020-11-16T22:20:49.328 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:20:49.330 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:49.359 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.058) 0:04:23.309 ******* 2020-11-16T22:20:49.810 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:20:49.812 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-16T22:20:49.842 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:49 +0000 (0:00:00.482) 0:04:23.791 ******* 2020-11-16T22:20:50.139 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:20:50.140 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:14:35 UTC", "ActiveEnterTimestampMonotonic": "15063268", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-tmpfiles-setup.service ntpdate.service systemd-journald.socket sntp.service -.mount ntpd.service basic.target tmp.mount sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:14:34 UTC", "AssertTimestampMonotonic": "14191592", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:14:34 UTC", "ConditionTimestampMonotonic": "14191567", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target ntpd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-16T22:20:50.143 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2438", "ExecMainStartTimestamp": "Mon 2020-11-16 22:14:35 UTC", "ExecMainStartTimestampMonotonic": "15014869", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:14:34 UTC", "InactiveExitTimestampMonotonic": "14239907", "InvocationID": "9df00eef0c39436daa318a7e0a8e4925", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-16T22:20:50.145 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2438", "MemoryAccounting": "yes", "MemoryCurrent": "2854912", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-16T22:20:50.146 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:14:35 UTC", "StateChangeTimestampMonotonic": "15063268", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:14:35 UTC", "WatchdogTimestampMonotonic": "15014882", "WatchdogUSec": "0"}} 2020-11-16T22:20:50.172 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.330) 0:04:24.122 ******* 2020-11-16T22:20:50.199 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:50.200 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:20:50.231 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.058) 0:04:24.181 ******* 2020-11-16T22:20:50.257 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:50.259 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-16T22:20:50.288 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.056) 0:04:24.238 ******* 2020-11-16T22:20:50.314 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:20:50.315 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:50.344 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.056) 0:04:24.294 ******* 2020-11-16T22:20:50.370 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:20:50.372 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:50.401 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.056) 0:04:24.351 ******* 2020-11-16T22:20:50.427 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:20:50.429 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:50.458 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.056) 0:04:24.407 ******* 2020-11-16T22:20:50.648 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:20:50.650 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:20:50.683 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:50 +0000 (0:00:00.225) 0:04:24.632 ******* 2020-11-16T22:20:51.154 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:20:51.156 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:20:51.186 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:51 +0000 (0:00:00.502) 0:04:25.135 ******* 2020-11-16T22:20:52.180 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:20:52.181 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:20:52.211 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:01.025) 0:04:26.161 ******* 2020-11-16T22:20:52.238 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:20:52.240 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:52.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.058) 0:04:26.219 ******* 2020-11-16T22:20:52.296 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:20:52.298 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:20:52.327 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.057) 0:04:26.276 ******* 2020-11-16T22:20:52.519 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:20:52.520 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-16T22:20:52.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.222) 0:04:26.499 ******* 2020-11-16T22:20:52.576 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:20:52.577 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:52.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.057) 0:04:26.557 ******* 2020-11-16T22:20:52.635 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:20:52.636 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:20:52.666 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.058) 0:04:26.615 ******* 2020-11-16T22:20:52.693 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:20:52.694 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:52.725 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.058) 0:04:26.674 ******* 2020-11-16T22:20:52.917 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:20:52.918 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:20:52.948 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:52 +0000 (0:00:00.223) 0:04:26.898 ******* 2020-11-16T22:20:53.143 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:20:53.145 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:20:53.174 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.225) 0:04:27.123 ******* 2020-11-16T22:20:53.200 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:20:53.201 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.231 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.056) 0:04:27.180 ******* 2020-11-16T22:20:53.267 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:20:53.268 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.278 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.280 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.310 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.079) 0:04:27.260 ******* 2020-11-16T22:20:53.348 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:20:53.350 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi194.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.360 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.361 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.371 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.403 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.093) 0:04:27.353 ******* 2020-11-16T22:20:53.430 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:20:53.432 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.059) 0:04:27.413 ******* 2020-11-16T22:20:53.490 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:20:53.491 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.521 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.057) 0:04:27.470 ******* 2020-11-16T22:20:53.548 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:20:53.549 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.060) 0:04:27.531 ******* 2020-11-16T22:20:53.608 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:20:53.609 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.638 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.057) 0:04:27.588 ******* 2020-11-16T22:20:53.667 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:20:53.668 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.699 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.060) 0:04:27.649 ******* 2020-11-16T22:20:53.726 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:53.727 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.059) 0:04:27.708 ******* 2020-11-16T22:20:53.785 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:20:53.787 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.818 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.060) 0:04:27.768 ******* 2020-11-16T22:20:53.846 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:20:53.847 INFO:teuthology.task.ansible.out:skipping: [smithi194.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:20:53.877 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:53 +0000 (0:00:00.058) 0:04:27.826 ******* 2020-11-16T22:20:54.100 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:20:54.102 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004784", "end": "2020-11-16 22:20:54.076791", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:20:54.072007", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:54.276 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003015", "end": "2020-11-16 22:20:54.255207", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:20:54.252192", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:54.452 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002786", "end": "2020-11-16 22:20:54.438261", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:20:54.435475", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:54.643 INFO:teuthology.task.ansible.out:ok: [smithi194.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.019781", "end": "2020-11-16 22:20:54.632441", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:20:54.612660", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:20:54.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.798) 0:04:28.625 ******* 2020-11-16T22:20:54.702 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:20:54.733 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:20:54 +0000 (0:00:00.057) 0:04:28.682 ******* 2020-11-16T22:20:54.925 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:20:54.927 INFO:teuthology.task.ansible.out:changed: [smithi194.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:20:55.015 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2020-11-16T22:20:55.017 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-11-16T22:20:55.019 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2020-11-16T22:20:55.020 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-11-16T22:20:55.022 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** 2020-11-16T22:20:55.023 INFO:teuthology.task.ansible.out:skipping: no hosts matched PLAY RECAP ********************************************************************* smithi122.front.sepia.ceph.com : ok=111 changed=31 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 smithi138.front.sepia.ceph.com : ok=111 changed=31 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 smithi173.front.sepia.ceph.com : ok=111 changed=31 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 smithi194.front.sepia.ceph.com : ok=111 changed=31 unreachable=0 failed=0 skipped=178 rescued=0 ignored=3 Monday 16 November 2020 22:20:55 +0000 (0:00:00.289) 0:04:28.972 ******* =============================================================================== testnode : Compile list of non-root partitions ------------------------ 103.49s common : nrpe - Remove SELinux policy package -------------------------- 16.06s common : nrpe - Load SELinux policy package ---------------------------- 12.29s users : Check keys_repo HEAD sha1 --------------------------------------- 7.81s Gathering Facts --------------------------------------------------------- 5.59s testnode : Ensure ceph dependency packages are not present. ------------- 4.24s common : Make sure smartmontools is installed --------------------------- 2.37s testnode : Zap all non-root disks --------------------------------------- 1.83s testnode : Ensure ceph packages are not present. ------------------------ 1.65s testnode : Rebuild rpmdb ------------------------------------------------ 1.59s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.41s testnode : Configure local mirrorlists ---------------------------------- 1.16s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.15s testnode : Install packages --------------------------------------------- 1.04s testnode : Install packages via pip ------------------------------------- 1.03s common : nrpe - 2020-11-16T22:20:55.025 INFO:teuthology.task.ansible.out: Set SELinux boolean nagios_run_sudo true ---------------- 0.81s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.80s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.76s Gathering Facts --------------------------------------------------------- 0.63s ansible-managed : Add authorized keys for the ansible user. ------------- 0.61s 2020-11-16T22:20:55.366 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2020-11-16T22:20:55.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:20:55.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:20:55.438 INFO:teuthology.orchestra.run.smithi138:> true 2020-11-16T22:20:55.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:20:55.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:20:55.597 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-16T22:20:55.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:20:55.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:20:55.761 INFO:teuthology.orchestra.run.smithi194:> true 2020-11-16T22:20:55.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:20:55.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:20:55.915 INFO:teuthology.orchestra.run.smithi122:> true 2020-11-16T22:20:56.001 INFO:teuthology.run_tasks:Running task clock... 2020-11-16T22:20:56.012 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-16T22:20:56.013 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:20:56.013 INFO:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:20:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-16T22:20:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-16T22:20:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2020-11-16T22:20:56.130 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2020-11-16T22:20:56.148 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-16T22:20:56.166 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-16T22:20:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:bash: ntpq: command not found 2020-11-16T22:20:56.227 INFO:teuthology.orchestra.run.smithi138.stdout:210 Number of sources = 4 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.228 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.229 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:20:56.230 INFO:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:20:56.270 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-16T22:20:56.289 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-16T22:20:56.334 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2020-11-16T22:20:56.348 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2020-11-16T22:20:56.366 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-16T22:20:56.383 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-16T22:20:56.446 INFO:teuthology.orchestra.run.smithi173.stderr:bash: ntpq: command not found 2020-11-16T22:20:56.449 INFO:teuthology.orchestra.run.smithi173.stdout:210 Number of sources = 4 2020-11-16T22:20:56.449 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-16T22:20:56.449 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2020-11-16T22:20:56.449 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.450 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.450 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.450 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.451 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:20:56.451 INFO:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:20:56.490 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-16T22:20:56.510 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-16T22:20:56.552 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2020-11-16T22:20:56.567 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2020-11-16T22:20:56.584 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-16T22:20:56.602 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-16T22:20:56.659 INFO:teuthology.orchestra.run.smithi194.stderr:bash: ntpq: command not found 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi194.stdout:210 Number of sources = 4 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:20:56.702 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-16T22:20:56.721 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-16T22:20:56.764 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2020-11-16T22:20:56.778 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2020-11-16T22:20:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-16T22:20:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi122.stderr:bash: ntpq: command not found 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi122.stdout:210 Number of sources = 4 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-16T22:20:56.885 INFO:teuthology.run_tasks:Running task install... 2020-11-16T22:20:56.893 DEBUG:teuthology.task.install:project ceph 2020-11-16T22:20:56.894 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}} 2020-11-16T22:20:56.894 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'} 2020-11-16T22:20:56.894 INFO:teuthology.task.install:Using flavor: basic 2020-11-16T22:20:56.900 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-16T22:20:56.900 INFO:teuthology.task.install:extra packages: [] 2020-11-16T22:20:56.901 INFO:teuthology.orchestra.run.smithi138:> sudo lsb_release -is 2020-11-16T22:20:56.941 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS 2020-11-16T22:20:56.942 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:56.943 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-16T22:20:56.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99', 'tag': None, 'wait_for_package': False} 2020-11-16T22:20:56.944 INFO:teuthology.orchestra.run.smithi138:> sudo lsb_release -is 2020-11-16T22:20:56.983 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS 2020-11-16T22:20:56.984 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:56.985 INFO:teuthology.orchestra.run.smithi194:> sudo lsb_release -is 2020-11-16T22:20:56.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99', 'tag': None, 'wait_for_package': False} 2020-11-16T22:20:56.985 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-16T22:20:57.020 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS 2020-11-16T22:20:57.021 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:20:57.027 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS 2020-11-16T22:20:57.027 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.027 INFO:teuthology.orchestra.run.smithi122:> sudo lsb_release -is 2020-11-16T22:20:57.028 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99', 'tag': None, 'wait_for_package': False} 2020-11-16T22:20:57.028 INFO:teuthology.orchestra.run.smithi194:> sudo lsb_release -is 2020-11-16T22:20:57.062 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS 2020-11-16T22:20:57.063 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:20:57.066 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS 2020-11-16T22:20:57.066 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.067 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99', 'tag': None, 'wait_for_package': False} 2020-11-16T22:20:57.067 INFO:teuthology.orchestra.run.smithi122:> sudo lsb_release -is 2020-11-16T22:20:57.102 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS 2020-11-16T22:20:57.103 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:20:57.143 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS 2020-11-16T22:20:57.145 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:20:57.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:20:57.195 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/ 2020-11-16T22:20:57.196 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7198.gb637909a 2020-11-16T22:20:57.222 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/ 2020-11-16T22:20:57.223 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7198.gb637909a 2020-11-16T22:20:57.261 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/ 2020-11-16T22:20:57.262 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7198.gb637909a 2020-11-16T22:20:57.310 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/ 2020-11-16T22:20:57.310 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7198.gb637909a 2020-11-16T22:20:57.332 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-16T22:20:57.333 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:20:57.333 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.345 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-16T22:20:57.346 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:20:57.346 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.347 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.359 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.361 INFO:teuthology.orchestra.run.smithi138.stderr:1+1 records in 2020-11-16T22:20:57.361 INFO:teuthology.orchestra.run.smithi138.stderr:1+1 records out 2020-11-16T22:20:57.361 INFO:teuthology.orchestra.run.smithi138.stderr:586 bytes copied, 3.9371e-05 s, 14.9 MB/s 2020-11-16T22:20:57.362 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-16T22:20:57.363 INFO:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b637909a9153a419511f943e4eee7800ad3bbb99/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-16T22:20:57.373 INFO:teuthology.orchestra.run.smithi173.stderr:1+1 records in 2020-11-16T22:20:57.373 INFO:teuthology.orchestra.run.smithi173.stderr:1+1 records out 2020-11-16T22:20:57.374 INFO:teuthology.orchestra.run.smithi173.stderr:586 bytes copied, 3.9103e-05 s, 15.0 MB/s 2020-11-16T22:20:57.374 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-16T22:20:57.375 INFO:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b637909a9153a419511f943e4eee7800ad3bbb99/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-16T22:20:57.384 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-16T22:20:57.385 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:20:57.385 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.398 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.411 INFO:teuthology.orchestra.run.smithi194.stderr:1+1 records in 2020-11-16T22:20:57.411 INFO:teuthology.orchestra.run.smithi194.stderr:1+1 records out 2020-11-16T22:20:57.411 INFO:teuthology.orchestra.run.smithi194.stderr:586 bytes copied, 3.5231e-05 s, 16.6 MB/s 2020-11-16T22:20:57.412 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-16T22:20:57.412 INFO:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b637909a9153a419511f943e4eee7800ad3bbb99/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-16T22:20:57.432 INFO:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-16T22:20:57.440 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/master/b637909a9153a419511f943e4eee7800ad3bbb99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-16T22:20:57.440 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:20:57.441 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.444 INFO:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-16T22:20:57.453 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-16T22:20:57.467 INFO:teuthology.orchestra.run.smithi122.stderr:1+1 records in 2020-11-16T22:20:57.467 INFO:teuthology.orchestra.run.smithi122.stderr:1+1 records out 2020-11-16T22:20:57.467 INFO:teuthology.orchestra.run.smithi122.stderr:586 bytes copied, 3.6487e-05 s, 16.1 MB/s 2020-11-16T22:20:57.468 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-16T22:20:57.468 INFO:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b637909a9153a419511f943e4eee7800ad3bbb99/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-16T22:20:57.479 INFO:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-16T22:20:57.513 INFO:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-16T22:20:57.525 INFO:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-16T22:20:57.559 INFO:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-16T22:20:57.580 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2020-11-16T22:20:57.581 INFO:teuthology.orchestra.run.smithi138:> sudo yum clean all 2020-11-16T22:20:57.591 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2020-11-16T22:20:57.592 INFO:teuthology.orchestra.run.smithi173:> sudo yum clean all 2020-11-16T22:20:57.615 INFO:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-16T22:20:57.626 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2020-11-16T22:20:57.627 INFO:teuthology.orchestra.run.smithi194:> sudo yum clean all 2020-11-16T22:20:57.682 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2020-11-16T22:20:57.683 INFO:teuthology.orchestra.run.smithi122:> sudo yum clean all 2020-11-16T22:20:57.859 INFO:teuthology.orchestra.run.smithi138.stdout:42 files removed 2020-11-16T22:20:57.868 INFO:teuthology.orchestra.run.smithi173.stdout:42 files removed 2020-11-16T22:20:57.887 INFO:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-16T22:20:57.898 INFO:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-16T22:20:57.903 INFO:teuthology.orchestra.run.smithi194.stdout:42 files removed 2020-11-16T22:20:57.932 INFO:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-16T22:20:57.959 INFO:teuthology.orchestra.run.smithi122.stdout:42 files removed 2020-11-16T22:20:57.988 INFO:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-16T22:20:58.481 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 288 kB/s | 72 kB 00:00 2020-11-16T22:20:58.488 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 294 kB/s | 72 kB 00:00 2020-11-16T22:20:58.522 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 293 kB/s | 72 kB 00:00 2020-11-16T22:20:58.585 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 289 kB/s | 72 kB 00:00 2020-11-16T22:20:58.720 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 70 kB/s | 15 kB 00:00 2020-11-16T22:20:58.725 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 71 kB/s | 15 kB 00:00 2020-11-16T22:20:58.759 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 71 kB/s | 15 kB 00:00 2020-11-16T22:20:58.821 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 71 kB/s | 15 kB 00:00 2020-11-16T22:20:58.949 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-16T22:20:58.953 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 7.0 kB/s | 1.4 kB 00:00 2020-11-16T22:20:58.985 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 7.0 kB/s | 1.4 kB 00:00 2020-11-16T22:20:59.050 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-16T22:20:59.281 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - AppStream 22 MB/s | 7.0 MB 00:00 2020-11-16T22:20:59.316 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - AppStream 22 MB/s | 7.0 MB 00:00 2020-11-16T22:20:59.347 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - AppStream 19 MB/s | 7.0 MB 00:00 2020-11-16T22:20:59.397 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - AppStream 21 MB/s | 7.0 MB 00:00 2020-11-16T22:21:00.464 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - Base 26 MB/s | 2.2 MB 00:00 2020-11-16T22:21:00.508 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - Base 23 MB/s | 2.2 MB 00:00 2020-11-16T22:21:00.534 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - Base 23 MB/s | 2.2 MB 00:00 2020-11-16T22:21:00.580 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-16T22:21:00.895 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-8 - Extras 88 kB/s | 5.5 kB 00:00 2020-11-16T22:21:00.953 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-8 - Extras 84 kB/s | 5.5 kB 00:00 2020-11-16T22:21:00.961 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-8 - Extras 80 kB/s | 5.5 kB 00:00 2020-11-16T22:21:01.006 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Extras 80 kB/s | 5.5 kB 00:00 2020-11-16T22:21:01.407 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 18 MB/s | 8.3 MB 00:00 2020-11-16T22:21:01.770 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 11 MB/s | 8.3 MB 00:00 2020-11-16T22:21:01.847 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 8.3 MB 00:00 2020-11-16T22:21:02.071 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 8.3 MB 00:01 2020-11-16T22:21:03.156 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 4.7 MB/s | 23 kB 00:00 2020-11-16T22:21:03.496 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 4.2 MB/s | 23 kB 00:00 2020-11-16T22:21:03.588 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 4.3 MB/s | 23 kB 00:00 2020-11-16T22:21:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 4.4 MB/s | 23 kB 00:00 2020-11-16T22:21:04.047 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.047 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.184 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2020-11-16T22:21:04.211 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2020-11-16T22:21:04.211 INFO:teuthology.orchestra.run.smithi138.stdout: Package ArchVersion Repository Size 2020-11-16T22:21:04.211 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2020-11-16T22:21:04.211 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2020-11-16T22:21:04.211 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_642:16.0.0-7198.gb637909a.el8 ceph 6.4 k 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_642:16.0.0-7198.gb637909a.el8 ceph 5.9 M 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 624 k 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_642:16.0.0-7198.gb637909a.el8 ceph 201 k 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_642:16.0.0-7198.gb637909a.el8 ceph 1.5 M 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_642:16.0.0-7198.gb637909a.el8 ceph 12 M 2020-11-16T22:21:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_642:16.0.0-7198.gb637909a.el8 ceph 47 M 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm x86_642:16.0.0-7198.gb637909a.el8 ceph 57 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 24 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_642:16.0.0-7198.gb637909a.el8 ceph 554 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 131 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_642:16.0.0-7198.gb637909a.el8 ceph 204 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_642:16.0.0-7198.gb637909a.el8 ceph 375 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_642:16.0.0-7198.gb637909a.el8 ceph 353 k 2020-11-16T22:21:04.213 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_642:16.0.0-7198.gb637909a.el8 ceph 113 k 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 87 k 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_642:16.0.0-7198.gb637909a.el8 ceph 3.1 M 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_642:16.0.0-7198.gb637909a.el8 ceph 168 k 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 66 k 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 3.2 M 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.6 M 2020-11-16T22:21:04.214 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 27 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-16T22:21:04.215 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_642:16.0.0-7198.gb637909a.el8 ceph 2.4 M 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 23 M 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_642:16.0.0-7198.gb637909a.el8 ceph 2.1 M 2020-11-16T22:21:04.216 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_642:16.0.0-7198.gb637909a.el8 ceph 4.9 M 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_642:16.0.0-7198.gb637909a.el8 ceph 18 M 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_642:16.0.0-7198.gb637909a.el8 ceph 25 k 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_642:16.0.0-7198.gb637909a.el8 ceph 480 k 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_642:16.0.0-7198.gb637909a.el8 ceph 40 k 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 73 k 2020-11-16T22:21:04.217 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 24 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 198 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.8 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-16T22:21:04.218 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-16T22:21:04.219 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-16T22:21:04.220 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-16T22:21:04.221 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-16T22:21:04.222 INFO:teuthology.orchestra.run.smithi138.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-16T22:21:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-16T22:21:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-16T22:21:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-16T22:21:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-16T22:21:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-16T22:21:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout:Downgrading: 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-16T22:21:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout:Install 126 Packages 2020-11-16T22:21:04.230 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2020-11-16T22:21:04.231 INFO:teuthology.orchestra.run.smithi138.stdout:Downgrade 5 Packages 2020-11-16T22:21:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:21:04.231 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 196 M 2020-11-16T22:21:04.231 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2020-11-16T22:21:04.268 INFO:teuthology.orchestra.run.smithi138.stdout:(1/133): krb5-libs-1.17-9.el8.x86_64.rpm 20 MB/s | 851 kB 00:00 2020-11-16T22:21:04.280 INFO:teuthology.orchestra.run.smithi138.stdout:(2/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 59 MB/s | 686 kB 00:00 2020-11-16T22:21:04.283 INFO:teuthology.orchestra.run.smithi138.stdout:(3/133): libkadm5-1.17-9.el8.x86_64.rpm 3.2 MB/s | 184 kB 00:00 2020-11-16T22:21:04.304 INFO:teuthology.orchestra.run.smithi138.stdout:(4/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 62 MB/s | 1.5 MB 00:00 2020-11-16T22:21:04.306 INFO:teuthology.orchestra.run.smithi138.stdout:(5/133): krb5-workstation-1.17-9.el8.x86_64.rpm 11 MB/s | 940 kB 00:00 2020-11-16T22:21:04.385 INFO:teuthology.orchestra.run.smithi138.stdout:(6/133): ceph-16.0.0-7198.gb637909a.el8.x86_64. 64 kB/s | 6.4 kB 00:00 2020-11-16T22:21:04.398 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.398 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.495 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.495 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.532 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2020-11-16T22:21:04.543 INFO:teuthology.orchestra.run.smithi138.stdout:(7/133): ceph-fuse-16.0.0-7198.gb637909a.el8.x8 3.8 MB/s | 624 kB 00:00 2020-11-16T22:21:04.549 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2020-11-16T22:21:04.549 INFO:teuthology.orchestra.run.smithi194.stdout: Package ArchVersion Repository Size 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_642:16.0.0-7198.gb637909a.el8 ceph 6.4 k 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_642:16.0.0-7198.gb637909a.el8 ceph 5.9 M 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 624 k 2020-11-16T22:21:04.550 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_642:16.0.0-7198.gb637909a.el8 ceph 201 k 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_642:16.0.0-7198.gb637909a.el8 ceph 1.5 M 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_642:16.0.0-7198.gb637909a.el8 ceph 12 M 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_642:16.0.0-7198.gb637909a.el8 ceph 47 M 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm x86_642:16.0.0-7198.gb637909a.el8 ceph 57 k 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 24 k 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_642:16.0.0-7198.gb637909a.el8 ceph 554 k 2020-11-16T22:21:04.551 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 131 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_642:16.0.0-7198.gb637909a.el8 ceph 204 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_642:16.0.0-7198.gb637909a.el8 ceph 375 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_642:16.0.0-7198.gb637909a.el8 ceph 353 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_642:16.0.0-7198.gb637909a.el8 ceph 113 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 87 k 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_642:16.0.0-7198.gb637909a.el8 ceph 3.1 M 2020-11-16T22:21:04.552 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_642:16.0.0-7198.gb637909a.el8 ceph 168 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 66 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 3.2 M 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.6 M 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 27 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-16T22:21:04.553 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2020-11-16T22:21:04.554 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_642:16.0.0-7198.gb637909a.el8 ceph 2.4 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 23 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_642:16.0.0-7198.gb637909a.el8 ceph 2.1 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_642:16.0.0-7198.gb637909a.el8 ceph 4.9 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_642:16.0.0-7198.gb637909a.el8 ceph 18 M 2020-11-16T22:21:04.555 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_642:16.0.0-7198.gb637909a.el8 ceph 25 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_642:16.0.0-7198.gb637909a.el8 ceph 480 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_642:16.0.0-7198.gb637909a.el8 ceph 40 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 73 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 24 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 198 k 2020-11-16T22:21:04.556 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.8 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-16T22:21:04.557 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-16T22:21:04.558 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-16T22:21:04.559 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-16T22:21:04.560 INFO:teuthology.orchestra.run.smithi194.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-16T22:21:04.561 INFO:teuthology.orchestra.run.smithi194.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-16T22:21:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-16T22:21:04.563 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-16T22:21:04.564 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-16T22:21:04.565 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-16T22:21:04.566 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-16T22:21:04.567 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout:Downgrading: 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-16T22:21:04.568 INFO:teuthology.orchestra.run.smithi194.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout:Install 126 Packages 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 2 Packages 2020-11-16T22:21:04.569 INFO:teuthology.orchestra.run.smithi194.stdout:Downgrade 5 Packages 2020-11-16T22:21:04.570 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:21:04.570 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 196 M 2020-11-16T22:21:04.570 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2020-11-16T22:21:04.584 INFO:teuthology.orchestra.run.smithi138.stdout:(8/133): ceph-immutable-object-cache-16.0.0-719 4.8 MB/s | 201 kB 00:00 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi194.stdout:(1/133): krb5-libs-1.17-9.el8.x86_64.rpm 19 MB/s | 851 kB 00:00 2020-11-16T22:21:04.613 INFO:teuthology.orchestra.run.smithi194.stdout:(2/133): libkadm5-1.17-9.el8.x86_64.rpm 3.8 MB/s | 184 kB 00:00 2020-11-16T22:21:04.622 INFO:teuthology.orchestra.run.smithi194.stdout:(3/133): krb5-workstation-1.17-9.el8.x86_64.rpm 16 MB/s | 940 kB 00:00 2020-11-16T22:21:04.627 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2020-11-16T22:21:04.629 INFO:teuthology.orchestra.run.smithi194.stdout:(4/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 35 MB/s | 686 kB 00:00 2020-11-16T22:21:04.644 INFO:teuthology.orchestra.run.smithi194.stdout:(5/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 47 MB/s | 1.5 MB 00:00 2020-11-16T22:21:04.648 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2020-11-16T22:21:04.648 INFO:teuthology.orchestra.run.smithi173.stdout: Package ArchVersion Repository Size 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_642:16.0.0-7198.gb637909a.el8 ceph 6.4 k 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_642:16.0.0-7198.gb637909a.el8 ceph 5.9 M 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 624 k 2020-11-16T22:21:04.649 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_642:16.0.0-7198.gb637909a.el8 ceph 201 k 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_642:16.0.0-7198.gb637909a.el8 ceph 1.5 M 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_642:16.0.0-7198.gb637909a.el8 ceph 12 M 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_642:16.0.0-7198.gb637909a.el8 ceph 47 M 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm x86_642:16.0.0-7198.gb637909a.el8 ceph 57 k 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 24 k 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_642:16.0.0-7198.gb637909a.el8 ceph 554 k 2020-11-16T22:21:04.650 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 131 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_642:16.0.0-7198.gb637909a.el8 ceph 204 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_642:16.0.0-7198.gb637909a.el8 ceph 375 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_642:16.0.0-7198.gb637909a.el8 ceph 353 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_642:16.0.0-7198.gb637909a.el8 ceph 113 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 87 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_642:16.0.0-7198.gb637909a.el8 ceph 3.1 M 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_642:16.0.0-7198.gb637909a.el8 ceph 168 k 2020-11-16T22:21:04.651 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 66 k 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 3.2 M 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.6 M 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 27 k 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-16T22:21:04.652 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_642:16.0.0-7198.gb637909a.el8 ceph 2.4 M 2020-11-16T22:21:04.653 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 23 M 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_642:16.0.0-7198.gb637909a.el8 ceph 2.1 M 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_642:16.0.0-7198.gb637909a.el8 ceph 4.9 M 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_642:16.0.0-7198.gb637909a.el8 ceph 18 M 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_642:16.0.0-7198.gb637909a.el8 ceph 25 k 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_642:16.0.0-7198.gb637909a.el8 ceph 480 k 2020-11-16T22:21:04.654 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_642:16.0.0-7198.gb637909a.el8 ceph 40 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 73 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 24 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 198 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.8 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-16T22:21:04.655 INFO:teuthology.orchestra.run.smithi173.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-16T22:21:04.656 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-16T22:21:04.657 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-16T22:21:04.658 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-16T22:21:04.659 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-16T22:21:04.660 INFO:teuthology.orchestra.run.smithi173.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-16T22:21:04.661 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-16T22:21:04.662 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-16T22:21:04.663 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-16T22:21:04.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-16T22:21:04.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-16T22:21:04.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-16T22:21:04.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-16T22:21:04.665 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-16T22:21:04.666 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout:Downgrading: 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-16T22:21:04.667 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2020-11-16T22:21:04.668 INFO:teuthology.orchestra.run.smithi173.stdout:Install 126 Packages 2020-11-16T22:21:04.669 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 2 Packages 2020-11-16T22:21:04.669 INFO:teuthology.orchestra.run.smithi173.stdout:Downgrade 5 Packages 2020-11-16T22:21:04.669 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:21:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-16T22:21:04.670 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 196 M 2020-11-16T22:21:04.670 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2020-11-16T22:21:04.712 INFO:teuthology.orchestra.run.smithi173.stdout:(1/133): krb5-workstation-1.17-9.el8.x86_64.rpm 19 MB/s | 940 kB 00:00 2020-11-16T22:21:04.714 INFO:teuthology.orchestra.run.smithi173.stdout:(2/133): libkadm5-1.17-9.el8.x86_64.rpm 3.6 MB/s | 184 kB 00:00 2020-11-16T22:21:04.725 INFO:teuthology.orchestra.run.smithi173.stdout:(3/133): krb5-libs-1.17-9.el8.x86_64.rpm 14 MB/s | 851 kB 00:00 2020-11-16T22:21:04.727 INFO:teuthology.orchestra.run.smithi173.stdout:(4/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 45 MB/s | 686 kB 00:00 2020-11-16T22:21:04.743 INFO:teuthology.orchestra.run.smithi173.stdout:(5/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 51 MB/s | 1.5 MB 00:00 2020-11-16T22:21:04.784 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2020-11-16T22:21:04.810 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2020-11-16T22:21:04.810 INFO:teuthology.orchestra.run.smithi122.stdout: Package ArchVersion Repository Size 2020-11-16T22:21:04.810 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2020-11-16T22:21:04.810 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_642:16.0.0-7198.gb637909a.el8 ceph 6.4 k 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_642:16.0.0-7198.gb637909a.el8 ceph 5.9 M 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 624 k 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_642:16.0.0-7198.gb637909a.el8 ceph 201 k 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_642:16.0.0-7198.gb637909a.el8 ceph 1.5 M 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_642:16.0.0-7198.gb637909a.el8 ceph 12 M 2020-11-16T22:21:04.811 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_642:16.0.0-7198.gb637909a.el8 ceph 47 M 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm x86_642:16.0.0-7198.gb637909a.el8 ceph 57 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 24 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_642:16.0.0-7198.gb637909a.el8 ceph 554 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_642:16.0.0-7198.gb637909a.el8 ceph 131 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_642:16.0.0-7198.gb637909a.el8 ceph 204 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_642:16.0.0-7198.gb637909a.el8 ceph 375 k 2020-11-16T22:21:04.812 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_642:16.0.0-7198.gb637909a.el8 ceph 353 k 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_642:16.0.0-7198.gb637909a.el8 ceph 113 k 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_642:16.0.0-7198.gb637909a.el8 ceph 87 k 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_642:16.0.0-7198.gb637909a.el8 ceph 3.1 M 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_642:16.0.0-7198.gb637909a.el8 ceph 168 k 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 66 k 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 3.2 M 2020-11-16T22:21:04.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.6 M 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 27 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-16T22:21:04.814 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_642:16.0.0-7198.gb637909a.el8 ceph 2.4 M 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 23 M 2020-11-16T22:21:04.815 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_642:16.0.0-7198.gb637909a.el8 ceph 2.1 M 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_642:16.0.0-7198.gb637909a.el8 ceph 4.9 M 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_642:16.0.0-7198.gb637909a.el8 ceph 18 M 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_642:16.0.0-7198.gb637909a.el8 ceph 25 k 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_642:16.0.0-7198.gb637909a.el8 ceph 480 k 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_642:16.0.0-7198.gb637909a.el8 ceph 3.7 M 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_642:16.0.0-7198.gb637909a.el8 ceph 40 k 2020-11-16T22:21:04.816 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_642:16.0.0-7198.gb637909a.el8 ceph 73 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 24 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 198 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch2:16.0.0-7198.gb637909a.el8 ceph-noarch 8.8 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-16T22:21:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-16T22:21:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-16T22:21:04.819 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-16T22:21:04.820 INFO:teuthology.orchestra.run.smithi122.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-16T22:21:04.821 INFO:teuthology.orchestra.run.smithi122.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-16T22:21:04.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-16T22:21:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-16T22:21:04.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-16T22:21:04.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-16T22:21:04.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-16T22:21:04.827 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout:Downgrading: 2020-11-16T22:21:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2020-11-16T22:21:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:Install 126 Packages 2020-11-16T22:21:04.830 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 2 Packages 2020-11-16T22:21:04.830 INFO:teuthology.orchestra.run.smithi122.stdout:Downgrade 5 Packages 2020-11-16T22:21:04.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:21:04.830 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 196 M 2020-11-16T22:21:04.831 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2020-11-16T22:21:04.841 INFO:teuthology.orchestra.run.smithi194.stdout:(6/133): ceph-16.0.0-7198.gb637909a.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2020-11-16T22:21:04.846 INFO:teuthology.orchestra.run.smithi138.stdout:(9/133): ceph-mds-16.0.0-7198.gb637909a.el8.x86 8.1 MB/s | 2.1 MB 00:00 2020-11-16T22:21:04.949 INFO:teuthology.orchestra.run.smithi173.stdout:(6/133): ceph-16.0.0-7198.gb637909a.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2020-11-16T22:21:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:(10/133): ceph-base-16.0.0-7198.gb637909a.el8.x 8.1 MB/s | 5.9 MB 00:00 2020-11-16T22:21:05.037 INFO:teuthology.orchestra.run.smithi138.stdout:(11/133): ceph-mgr-16.0.0-7198.gb637909a.el8.x8 8.0 MB/s | 1.5 MB 00:00 2020-11-16T22:21:05.423 INFO:teuthology.orchestra.run.smithi122.stdout:(1/133): krb5-libs-1.17-9.el8.x86_64.rpm 1.4 MB/s | 851 kB 00:00 2020-11-16T22:21:05.431 INFO:teuthology.orchestra.run.smithi122.stdout:(2/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 79 MB/s | 686 kB 00:00 2020-11-16T22:21:05.449 INFO:teuthology.orchestra.run.smithi122.stdout:(3/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 84 MB/s | 1.5 MB 00:00 2020-11-16T22:21:05.657 INFO:teuthology.orchestra.run.smithi138.stdout:(12/133): ceph-mon-16.0.0-7198.gb637909a.el8.x8 7.9 MB/s | 4.9 MB 00:00 2020-11-16T22:21:05.689 INFO:teuthology.orchestra.run.smithi194.stdout:(7/133): ceph-fuse-16.0.0-7198.gb637909a.el8.x8 736 kB/s | 624 kB 00:00 2020-11-16T22:21:05.695 INFO:teuthology.orchestra.run.smithi122.stdout:(4/133): ceph-16.0.0-7198.gb637909a.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2020-11-16T22:21:05.775 INFO:teuthology.orchestra.run.smithi122.stdout:(5/133): krb5-workstation-1.17-9.el8.x86_64.rpm 991 kB/s | 940 kB 00:00 2020-11-16T22:21:05.840 INFO:teuthology.orchestra.run.smithi173.stdout:(7/133): ceph-fuse-16.0.0-7198.gb637909a.el8.x8 701 kB/s | 624 kB 00:00 2020-11-16T22:21:05.905 INFO:teuthology.orchestra.run.smithi194.stdout:(8/133): ceph-immutable-object-cache-16.0.0-719 931 kB/s | 201 kB 00:00 2020-11-16T22:21:06.068 INFO:teuthology.orchestra.run.smithi173.stdout:(8/133): ceph-immutable-object-cache-16.0.0-719 882 kB/s | 201 kB 00:00 2020-11-16T22:21:06.416 INFO:teuthology.orchestra.run.smithi122.stdout:(6/133): libkadm5-1.17-9.el8.x86_64.rpm 116 kB/s | 184 kB 00:01 2020-11-16T22:21:07.030 INFO:teuthology.orchestra.run.smithi122.stdout:(7/133): ceph-fuse-16.0.0-7198.gb637909a.el8.x8 1.0 MB/s | 624 kB 00:00 2020-11-16T22:21:07.104 INFO:teuthology.orchestra.run.smithi122.stdout:(8/133): ceph-immutable-object-cache-16.0.0-719 2.7 MB/s | 201 kB 00:00 2020-11-16T22:21:07.388 INFO:teuthology.orchestra.run.smithi138.stdout:(13/133): ceph-common-16.0.0-7198.gb637909a.el8 7.4 MB/s | 23 MB 00:03 2020-11-16T22:21:07.493 INFO:teuthology.orchestra.run.smithi138.stdout:(14/133): ceph-radosgw-16.0.0-7198.gb637909a.el 6.4 MB/s | 12 MB 00:01 2020-11-16T22:21:07.493 INFO:teuthology.orchestra.run.smithi138.stdout:(15/133): ceph-selinux-16.0.0-7198.gb637909a.el 238 kB/s | 25 kB 00:00 2020-11-16T22:21:07.539 INFO:teuthology.orchestra.run.smithi122.stdout:(9/133): ceph-mds-16.0.0-7198.gb637909a.el8.x86 4.9 MB/s | 2.1 MB 00:00 2020-11-16T22:21:07.569 INFO:teuthology.orchestra.run.smithi138.stdout:(16/133): cephadm-16.0.0-7198.gb637909a.el8.x86 756 kB/s | 57 kB 00:00 2020-11-16T22:21:07.581 INFO:teuthology.orchestra.run.smithi194.stdout:(9/133): ceph-mds-16.0.0-7198.gb637909a.el8.x86 1.3 MB/s | 2.1 MB 00:01 2020-11-16T22:21:07.677 INFO:teuthology.orchestra.run.smithi138.stdout:(17/133): libcephfs-devel-16.0.0-7198.gb637909a 218 kB/s | 24 kB 00:00 2020-11-16T22:21:07.886 INFO:teuthology.orchestra.run.smithi122.stdout:(10/133): ceph-mgr-16.0.0-7198.gb637909a.el8.x8 4.4 MB/s | 1.5 MB 00:00 2020-11-16T22:21:07.888 INFO:teuthology.orchestra.run.smithi138.stdout:(18/133): libcephfs2-16.0.0-7198.gb637909a.el8. 2.6 MB/s | 554 kB 00:00 2020-11-16T22:21:07.961 INFO:teuthology.orchestra.run.smithi138.stdout:(19/133): librados-devel-16.0.0-7198.gb637909a. 1.8 MB/s | 131 kB 00:00 2020-11-16T22:21:08.094 INFO:teuthology.orchestra.run.smithi173.stdout:(9/133): ceph-mds-16.0.0-7198.gb637909a.el8.x86 1.0 MB/s | 2.1 MB 00:02 2020-11-16T22:21:08.104 INFO:teuthology.orchestra.run.smithi138.stdout:(20/133): ceph-osd-16.0.0-7198.gb637909a.el8.x8 5.8 MB/s | 18 MB 00:03 2020-11-16T22:21:08.108 INFO:teuthology.orchestra.run.smithi138.stdout:(21/133): libradosstriper1-16.0.0-7198.gb637909 3.2 MB/s | 480 kB 00:00 2020-11-16T22:21:08.178 INFO:teuthology.orchestra.run.smithi138.stdout:(22/133): python3-ceph-argparse-16.0.0-7198.gb6 578 kB/s | 40 kB 00:00 2020-11-16T22:21:08.240 INFO:teuthology.orchestra.run.smithi138.stdout:(23/133): python3-ceph-common-16.0.0-7198.gb637 1.1 MB/s | 73 kB 00:00 2020-11-16T22:21:08.310 INFO:teuthology.orchestra.run.smithi138.stdout:(24/133): python3-cephfs-16.0.0-7198.gb637909a. 2.9 MB/s | 204 kB 00:00 2020-11-16T22:21:08.453 INFO:teuthology.orchestra.run.smithi138.stdout:(25/133): python3-rados-16.0.0-7198.gb637909a.e 2.6 MB/s | 375 kB 00:00 2020-11-16T22:21:08.587 INFO:teuthology.orchestra.run.smithi138.stdout:(26/133): python3-rbd-16.0.0-7198.gb637909a.el8 2.6 MB/s | 353 kB 00:00 2020-11-16T22:21:08.644 INFO:teuthology.orchestra.run.smithi194.stdout:(10/133): ceph-mgr-16.0.0-7198.gb637909a.el8.x8 1.4 MB/s | 1.5 MB 00:01 2020-11-16T22:21:08.651 INFO:teuthology.orchestra.run.smithi138.stdout:(27/133): python3-rgw-16.0.0-7198.gb637909a.el8 1.7 MB/s | 113 kB 00:00 2020-11-16T22:21:08.718 INFO:teuthology.orchestra.run.smithi138.stdout:(28/133): rbd-fuse-16.0.0-7198.gb637909a.el8.x8 1.3 MB/s | 87 kB 00:00 2020-11-16T22:21:08.869 INFO:teuthology.orchestra.run.smithi122.stdout:(11/133): ceph-mon-16.0.0-7198.gb637909a.el8.x8 5.0 MB/s | 4.9 MB 00:00 2020-11-16T22:21:08.953 INFO:teuthology.orchestra.run.smithi138.stdout:(29/133): librgw2-16.0.0-7198.gb637909a.el8.x86 4.3 MB/s | 3.7 MB 00:00 2020-11-16T22:21:09.026 INFO:teuthology.orchestra.run.smithi138.stdout:(30/133): rbd-nbd-16.0.0-7198.gb637909a.el8.x86 2.3 MB/s | 168 kB 00:00 2020-11-16T22:21:09.091 INFO:teuthology.orchestra.run.smithi138.stdout:(31/133): ceph-grafana-dashboards-16.0.0-7198.g 368 kB/s | 24 kB 00:00 2020-11-16T22:21:09.157 INFO:teuthology.orchestra.run.smithi138.stdout:(32/133): ceph-mgr-cephadm-16.0.0-7198.gb637909 1.0 MB/s | 66 kB 00:00 2020-11-16T22:21:09.161 INFO:teuthology.orchestra.run.smithi173.stdout:(10/133): ceph-base-16.0.0-7198.gb637909a.el8.x 1.3 MB/s | 5.9 MB 00:04 2020-11-16T22:21:09.372 INFO:teuthology.orchestra.run.smithi194.stdout:(11/133): ceph-base-16.0.0-7198.gb637909a.el8.x 1.2 MB/s | 5.9 MB 00:04 2020-11-16T22:21:09.423 INFO:teuthology.orchestra.run.smithi138.stdout:(33/133): rbd-mirror-16.0.0-7198.gb637909a.el8. 4.5 MB/s | 3.1 MB 00:00 2020-11-16T22:21:09.476 INFO:teuthology.orchestra.run.smithi173.stdout:(11/133): ceph-mgr-16.0.0-7198.gb637909a.el8.x8 1.1 MB/s | 1.5 MB 00:01 2020-11-16T22:21:09.980 INFO:teuthology.orchestra.run.smithi138.stdout:(34/133): ceph-mgr-dashboard-16.0.0-7198.gb6379 3.8 MB/s | 3.2 MB 00:00 2020-11-16T22:21:10.061 INFO:teuthology.orchestra.run.smithi138.stdout:(35/133): ceph-mgr-modules-core-16.0.0-7198.gb6 2.4 MB/s | 198 kB 00:00 2020-11-16T22:21:10.132 INFO:teuthology.orchestra.run.smithi138.stdout:(36/133): ceph-mgr-rook-16.0.0-7198.gb637909a.e 384 kB/s | 27 kB 00:00 2020-11-16T22:21:10.221 INFO:teuthology.orchestra.run.smithi138.stdout:(37/133): ceph-prometheus-alerts-16.0.0-7198.gb 100 kB/s | 8.8 kB 00:00 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi138.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 18 MB/s | 713 kB 00:00 2020-11-16T22:21:10.269 INFO:teuthology.orchestra.run.smithi138.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 77 MB/s | 688 kB 00:00 2020-11-16T22:21:10.274 INFO:teuthology.orchestra.run.smithi138.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 70 MB/s | 320 kB 00:00 2020-11-16T22:21:10.276 INFO:teuthology.orchestra.run.smithi138.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 61 MB/s | 161 kB 00:00 2020-11-16T22:21:10.281 INFO:teuthology.orchestra.run.smithi138.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 73 MB/s | 353 kB 00:00 2020-11-16T22:21:10.290 INFO:teuthology.orchestra.run.smithi138.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 80 MB/s | 709 kB 00:00 2020-11-16T22:21:10.294 INFO:teuthology.orchestra.run.smithi138.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2020-11-16T22:21:10.297 INFO:teuthology.orchestra.run.smithi138.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 63 MB/s | 188 kB 00:00 2020-11-16T22:21:10.347 INFO:teuthology.orchestra.run.smithi138.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 86 MB/s | 4.3 MB 00:00 2020-11-16T22:21:10.399 INFO:teuthology.orchestra.run.smithi138.stdout:(47/133): openblas-threads-0.3.3-2.el8.x86_64.r 86 MB/s | 4.4 MB 00:00 2020-11-16T22:21:10.455 INFO:teuthology.orchestra.run.smithi138.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 86 MB/s | 4.8 MB 00:00 2020-11-16T22:21:10.458 INFO:teuthology.orchestra.run.smithi138.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 63 MB/s | 214 kB 00:00 2020-11-16T22:21:10.465 INFO:teuthology.orchestra.run.smithi138.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 77 MB/s | 538 kB 00:00 2020-11-16T22:21:10.466 INFO:teuthology.orchestra.run.smithi138.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 25 MB/s | 27 kB 00:00 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi138.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 19 MB/s | 20 kB 00:00 2020-11-16T22:21:10.485 INFO:teuthology.orchestra.run.smithi138.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 84 MB/s | 1.5 MB 00:00 2020-11-16T22:21:10.488 INFO:teuthology.orchestra.run.smithi138.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 62 MB/s | 157 kB 00:00 2020-11-16T22:21:10.489 INFO:teuthology.orchestra.run.smithi138.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 30 MB/s | 39 kB 00:00 2020-11-16T22:21:10.533 INFO:teuthology.orchestra.run.smithi138.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 86 MB/s | 3.7 MB 00:00 2020-11-16T22:21:10.537 INFO:teuthology.orchestra.run.smithi138.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 64 MB/s | 225 kB 00:00 2020-11-16T22:21:10.538 INFO:teuthology.orchestra.run.smithi138.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 34 MB/s | 44 kB 00:00 2020-11-16T22:21:10.541 INFO:teuthology.orchestra.run.smithi138.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 60 MB/s | 126 kB 00:00 2020-11-16T22:21:10.543 INFO:teuthology.orchestra.run.smithi138.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 55 MB/s | 110 kB 00:00 2020-11-16T22:21:10.544 INFO:teuthology.orchestra.run.smithi138.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 39 MB/s | 54 kB 00:00 2020-11-16T22:21:10.785 INFO:teuthology.orchestra.run.smithi138.stdout:(62/133): python3-scipy-1.0.0-20.module_el8.1.0 61 MB/s | 15 MB 00:00 2020-11-16T22:21:10.787 INFO:teuthology.orchestra.run.smithi138.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 18 MB/s | 27 kB 00:00 2020-11-16T22:21:10.793 INFO:teuthology.orchestra.run.smithi138.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 73 MB/s | 457 kB 00:00 2020-11-16T22:21:10.797 INFO:teuthology.orchestra.run.smithi138.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 67 MB/s | 298 kB 00:00 2020-11-16T22:21:10.799 INFO:teuthology.orchestra.run.smithi138.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 37 MB/s | 54 kB 00:00 2020-11-16T22:21:10.801 INFO:teuthology.orchestra.run.smithi138.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 43 MB/s | 82 kB 00:00 2020-11-16T22:21:10.802 INFO:teuthology.orchestra.run.smithi138.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 18 MB/s | 21 kB 00:00 2020-11-16T22:21:10.804 INFO:teuthology.orchestra.run.smithi138.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 31 MB/s | 48 kB 00:00 2020-11-16T22:21:10.811 INFO:teuthology.orchestra.run.smithi138.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 76 MB/s | 548 kB 00:00 2020-11-16T22:21:10.812 INFO:teuthology.orchestra.run.smithi138.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 16 MB/s | 19 kB 00:00 2020-11-16T22:21:10.814 INFO:teuthology.orchestra.run.smithi138.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 28 MB/s | 38 kB 00:00 2020-11-16T22:21:10.822 INFO:teuthology.orchestra.run.smithi138.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 77 MB/s | 638 kB 00:00 2020-11-16T22:21:10.825 INFO:teuthology.orchestra.run.smithi138.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 59 MB/s | 169 kB 00:00 2020-11-16T22:21:10.826 INFO:teuthology.orchestra.run.smithi138.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 32 MB/s | 47 kB 00:00 2020-11-16T22:21:10.830 INFO:teuthology.orchestra.run.smithi138.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 65 MB/s | 199 kB 00:00 2020-11-16T22:21:10.832 INFO:teuthology.orchestra.run.smithi138.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 45 MB/s | 86 kB 00:00 2020-11-16T22:21:10.834 INFO:teuthology.orchestra.run.smithi138.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 48 MB/s | 95 kB 00:00 2020-11-16T22:21:10.835 INFO:teuthology.orchestra.run.smithi138.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 17 MB/s | 18 kB 00:00 2020-11-16T22:21:10.839 INFO:teuthology.orchestra.run.smithi138.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 68 MB/s | 249 kB 00:00 2020-11-16T22:21:10.866 INFO:teuthology.orchestra.run.smithi138.stdout:(81/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 85 MB/s | 2.3 MB 00:00 2020-11-16T22:21:10.873 INFO:teuthology.orchestra.run.smithi138.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 77 MB/s | 605 kB 00:00 2020-11-16T22:21:10.877 INFO:teuthology.orchestra.run.smithi138.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 65 MB/s | 228 kB 00:00 2020-11-16T22:21:10.881 INFO:teuthology.orchestra.run.smithi138.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 65 MB/s | 220 kB 00:00 2020-11-16T22:21:10.884 INFO:teuthology.orchestra.run.smithi138.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 61 MB/s | 195 kB 00:00 2020-11-16T22:21:10.885 INFO:teuthology.orchestra.run.smithi138.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 31 MB/s | 43 kB 00:00 2020-11-16T22:21:10.888 INFO:teuthology.orchestra.run.smithi138.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 59 MB/s | 155 kB 00:00 2020-11-16T22:21:10.889 INFO:teuthology.orchestra.run.smithi138.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 27 MB/s | 34 kB 00:00 2020-11-16T22:21:10.892 INFO:teuthology.orchestra.run.smithi138.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 52 MB/s | 123 kB 00:00 2020-11-16T22:21:10.893 INFO:teuthology.orchestra.run.smithi138.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 34 MB/s | 43 kB 00:00 2020-11-16T22:21:10.896 INFO:teuthology.orchestra.run.smithi138.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 62 MB/s | 176 kB 00:00 2020-11-16T22:21:10.898 INFO:teuthology.orchestra.run.smithi138.stdout:(92/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 42 MB/s | 69 kB 00:00 2020-11-16T22:21:10.900 INFO:teuthology.orchestra.run.smithi138.stdout:(93/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 35 MB/s | 56 kB 00:00 2020-11-16T22:21:11.063 INFO:teuthology.orchestra.run.smithi138.stdout:(94/133): fmt-6.2.1-1.el8.x86_64.rpm 713 kB/s | 116 kB 00:00 2020-11-16T22:21:11.103 INFO:teuthology.orchestra.run.smithi138.stdout:(95/133): gperftools-libs-2.7-6.el8.x86_64.rpm 7.7 MB/s | 315 kB 00:00 2020-11-16T22:21:11.124 INFO:teuthology.orchestra.run.smithi138.stdout:(96/133): leveldb-1.22-1.el8.x86_64.rpm 8.6 MB/s | 181 kB 00:00 2020-11-16T22:21:11.144 INFO:teuthology.orchestra.run.smithi138.stdout:(97/133): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2020-11-16T22:21:11.167 INFO:teuthology.orchestra.run.smithi138.stdout:(98/133): ceph-mgr-diskprediction-local-16.0.0- 4.9 MB/s | 8.6 MB 00:01 2020-11-16T22:21:11.168 INFO:teuthology.orchestra.run.smithi138.stdout:(99/133): libunwind-1.3.1-3.el8.x86_64.rpm 3.0 MB/s | 75 kB 00:00 2020-11-16T22:21:11.189 INFO:teuthology.orchestra.run.smithi138.stdout:(100/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.1 MB/s | 44 kB 00:00 2020-11-16T22:21:11.210 INFO:teuthology.orchestra.run.smithi138.stdout:(101/133): python3-beautifulsoup4-4.6.3-2.el8.1 8.7 MB/s | 185 kB 00:00 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi138.stdout:(102/133): python3-cachetools-3.1.1-4.el8.noarc 1.6 MB/s | 33 kB 00:00 2020-11-16T22:21:11.249 INFO:teuthology.orchestra.run.smithi138.stdout:(103/133): python3-certifi-2018.10.15-7.el8.noa 838 kB/s | 16 kB 00:00 2020-11-16T22:21:11.268 INFO:teuthology.orchestra.run.smithi138.stdout:(104/133): python3-apipkg-1.5-6.el8.noarch.rpm 211 kB/s | 21 kB 00:00 2020-11-16T22:21:11.269 INFO:teuthology.orchestra.run.smithi138.stdout:(105/133): python3-cheroot-8.2.1-1.el8.noarch.r 6.9 MB/s | 145 kB 00:00 2020-11-16T22:21:11.289 INFO:teuthology.orchestra.run.smithi138.stdout:(106/133): python3-cssselect-0.9.2-10.el8.noarc 2.0 MB/s | 40 kB 00:00 2020-11-16T22:21:11.327 INFO:teuthology.orchestra.run.smithi138.stdout:(107/133): python3-execnet-1.7.1-1.el8.noarch.r 9.3 MB/s | 355 kB 00:00 2020-11-16T22:21:11.349 INFO:teuthology.orchestra.run.smithi138.stdout:(108/133): python3-google-auth-1.1.1-10.el8.noa 3.7 MB/s | 81 kB 00:00 2020-11-16T22:21:11.365 INFO:teuthology.orchestra.run.smithi138.stdout:(109/133): python3-cherrypy-18.4.0-1.el8.noarch 3.9 MB/s | 384 kB 00:00 2020-11-16T22:21:11.368 INFO:teuthology.orchestra.run.smithi138.stdout:(110/133): python3-jaraco-6.2-6.el8.noarch.rpm 570 kB/s | 11 kB 00:00 2020-11-16T22:21:11.385 INFO:teuthology.orchestra.run.smithi138.stdout:(111/133): python3-jaraco-functools-2.0-4.el8.n 888 kB/s | 18 kB 00:00 2020-11-16T22:21:11.405 INFO:teuthology.orchestra.run.smithi138.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.4 MB/s | 49 kB 00:00 2020-11-16T22:21:11.411 INFO:teuthology.orchestra.run.smithi122.stdout:(12/133): ceph-base-16.0.0-7198.gb637909a.el8.x 1.0 MB/s | 5.9 MB 00:05 2020-11-16T22:21:11.425 INFO:teuthology.orchestra.run.smithi138.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-16T22:21:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:(114/133): python3-kubernetes-11.0.0-2.el8.noar 15 MB/s | 1.2 MB 00:00 2020-11-16T22:21:11.449 INFO:teuthology.orchestra.run.smithi138.stdout:(115/133): python3-pecan-1.3.2-9.el8.noarch.rpm 11 MB/s | 283 kB 00:00 2020-11-16T22:21:11.468 INFO:teuthology.orchestra.run.smithi138.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 781 kB/s | 16 kB 00:00 2020-11-16T22:21:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.4 MB/s | 48 kB 00:00 2020-11-16T22:21:11.489 INFO:teuthology.orchestra.run.smithi138.stdout:(118/133): python3-repoze-lru-0.7-6.el8.noarch. 1.6 MB/s | 34 kB 00:00 2020-11-16T22:21:11.490 INFO:teuthology.orchestra.run.smithi138.stdout:(119/133): python3-routes-2.4.1-12.el8.noarch.r 9.1 MB/s | 196 kB 00:00 2020-11-16T22:21:11.509 INFO:teuthology.orchestra.run.smithi138.stdout:(120/133): python3-rsa-3.4.2-15.el8.noarch.rpm 3.5 MB/s | 73 kB 00:00 2020-11-16T22:21:11.510 INFO:teuthology.orchestra.run.smithi138.stdout:(121/133): python3-simplegeneric-0.8.1-17.el8.n 958 kB/s | 19 kB 00:00 2020-11-16T22:21:11.529 INFO:teuthology.orchestra.run.smithi138.stdout:(122/133): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-11-16T22:21:11.530 INFO:teuthology.orchestra.run.smithi138.stdout:(123/133): python3-tempora-1.14.1-5.el8.noarch. 1.5 MB/s | 29 kB 00:00 2020-11-16T22:21:11.549 INFO:teuthology.orchestra.run.smithi138.stdout:(124/133): python3-trustme-0.6.0-4.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2020-11-16T22:21:11.552 INFO:teuthology.orchestra.run.smithi138.stdout:(125/133): python3-waitress-1.2.1-2.el8.1.noarc 11 MB/s | 241 kB 00:00 2020-11-16T22:21:11.570 INFO:teuthology.orchestra.run.smithi138.stdout:(126/133): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-11-16T22:21:11.573 INFO:teuthology.orchestra.run.smithi138.stdout:(127/133): python3-websocket-client-0.56.0-5.el 2.9 MB/s | 61 kB 00:00 2020-11-16T22:21:11.590 INFO:teuthology.orchestra.run.smithi138.stdout:(128/133): python3-webtest-2.0.33-1.el8.noarch. 4.2 MB/s | 85 kB 00:00 2020-11-16T22:21:11.593 INFO:teuthology.orchestra.run.smithi138.stdout:(129/133): python3-zc-lockfile-2.0-2.el8.noarch 1.1 MB/s | 23 kB 00:00 2020-11-16T22:21:11.610 INFO:teuthology.orchestra.run.smithi138.stdout:(130/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.4 MB/s | 69 kB 00:00 2020-11-16T22:21:11.727 INFO:teuthology.orchestra.run.smithi194.stdout:(12/133): ceph-mon-16.0.0-7198.gb637909a.el8.x8 1.6 MB/s | 4.9 MB 00:03 2020-11-16T22:21:12.029 INFO:teuthology.orchestra.run.smithi173.stdout:(12/133): ceph-mon-16.0.0-7198.gb637909a.el8.x8 1.7 MB/s | 4.9 MB 00:02 2020-11-16T22:21:12.146 INFO:teuthology.orchestra.run.smithi138.stdout:(131/133): librbd1-16.0.0-7198.gb637909a.el8.x8 4.4 MB/s | 2.4 MB 00:00 2020-11-16T22:21:12.212 INFO:teuthology.orchestra.run.smithi122.stdout:(13/133): ceph-osd-16.0.0-7198.gb637909a.el8.x8 5.3 MB/s | 18 MB 00:03 2020-11-16T22:21:12.267 INFO:teuthology.orchestra.run.smithi122.stdout:(14/133): ceph-selinux-16.0.0-7198.gb637909a.el 453 kB/s | 25 kB 00:00 2020-11-16T22:21:12.920 INFO:teuthology.orchestra.run.smithi138.stdout:(132/133): librados2-16.0.0-7198.gb637909a.el8. 2.8 MB/s | 3.7 MB 00:01 2020-11-16T22:21:16.089 INFO:teuthology.orchestra.run.smithi194.stdout:(13/133): ceph-common-16.0.0-7198.gb637909a.el8 2.0 MB/s | 23 MB 00:11 2020-11-16T22:21:16.163 INFO:teuthology.orchestra.run.smithi194.stdout:(14/133): ceph-selinux-16.0.0-7198.gb637909a.el 338 kB/s | 25 kB 00:00 2020-11-16T22:21:16.652 INFO:teuthology.orchestra.run.smithi138.stdout:(133/133): ceph-test-16.0.0-7198.gb637909a.el8. 5.1 MB/s | 47 MB 00:09 2020-11-16T22:21:16.656 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2020-11-16T22:21:16.657 INFO:teuthology.orchestra.run.smithi138.stdout:Total 16 MB/s | 196 MB 00:12 2020-11-16T22:21:16.720 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2020-11-16T22:21:16.882 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2020-11-16T22:21:16.882 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2020-11-16T22:21:16.923 INFO:teuthology.orchestra.run.smithi173.stdout:(13/133): ceph-radosgw-16.0.0-7198.gb637909a.el 2.4 MB/s | 12 MB 00:04 2020-11-16T22:21:16.962 INFO:teuthology.orchestra.run.smithi194.stdout:(15/133): ceph-radosgw-16.0.0-7198.gb637909a.el 2.2 MB/s | 12 MB 00:05 2020-11-16T22:21:16.974 INFO:teuthology.orchestra.run.smithi173.stdout:(14/133): ceph-selinux-16.0.0-7198.gb637909a.el 495 kB/s | 25 kB 00:00 2020-11-16T22:21:17.025 INFO:teuthology.orchestra.run.smithi194.stdout:(16/133): cephadm-16.0.0-7198.gb637909a.el8.x86 906 kB/s | 57 kB 00:00 2020-11-16T22:21:17.093 INFO:teuthology.orchestra.run.smithi194.stdout:(17/133): libcephfs-devel-16.0.0-7198.gb637909a 351 kB/s | 24 kB 00:00 2020-11-16T22:21:17.209 INFO:teuthology.orchestra.run.smithi173.stdout:(15/133): ceph-common-16.0.0-7198.gb637909a.el8 1.8 MB/s | 23 MB 00:12 2020-11-16T22:21:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:(15/133): ceph-radosgw-16.0.0-7198.gb637909a.el 2.0 MB/s | 12 MB 00:05 2020-11-16T22:21:17.280 INFO:teuthology.orchestra.run.smithi173.stdout:(16/133): cephadm-16.0.0-7198.gb637909a.el8.x86 814 kB/s | 57 kB 00:00 2020-11-16T22:21:17.292 INFO:teuthology.orchestra.run.smithi194.stdout:(18/133): libcephfs2-16.0.0-7198.gb637909a.el8. 2.7 MB/s | 554 kB 00:00 2020-11-16T22:21:17.327 INFO:teuthology.orchestra.run.smithi122.stdout:(16/133): cephadm-16.0.0-7198.gb637909a.el8.x86 922 kB/s | 57 kB 00:00 2020-11-16T22:21:17.341 INFO:teuthology.orchestra.run.smithi173.stdout:(17/133): libcephfs-devel-16.0.0-7198.gb637909a 387 kB/s | 24 kB 00:00 2020-11-16T22:21:17.355 INFO:teuthology.orchestra.run.smithi194.stdout:(19/133): librados-devel-16.0.0-7198.gb637909a. 2.0 MB/s | 131 kB 00:00 2020-11-16T22:21:17.388 INFO:teuthology.orchestra.run.smithi122.stdout:(17/133): libcephfs-devel-16.0.0-7198.gb637909a 384 kB/s | 24 kB 00:00 2020-11-16T22:21:17.552 INFO:teuthology.orchestra.run.smithi194.stdout:(20/133): libradosstriper1-16.0.0-7198.gb637909 2.4 MB/s | 480 kB 00:00 2020-11-16T22:21:17.601 INFO:teuthology.orchestra.run.smithi122.stdout:(18/133): libcephfs2-16.0.0-7198.gb637909a.el8. 2.6 MB/s | 554 kB 00:00 2020-11-16T22:21:17.659 INFO:teuthology.orchestra.run.smithi122.stdout:(19/133): librados-devel-16.0.0-7198.gb637909a. 2.2 MB/s | 131 kB 00:00 2020-11-16T22:21:17.663 INFO:teuthology.orchestra.run.smithi173.stdout:(18/133): ceph-osd-16.0.0-7198.gb637909a.el8.x8 2.2 MB/s | 18 MB 00:08 2020-11-16T22:21:17.665 INFO:teuthology.orchestra.run.smithi173.stdout:(19/133): libcephfs2-16.0.0-7198.gb637909a.el8. 1.7 MB/s | 554 kB 00:00 2020-11-16T22:21:17.744 INFO:teuthology.orchestra.run.smithi173.stdout:(20/133): librados-devel-16.0.0-7198.gb637909a. 1.6 MB/s | 131 kB 00:00 2020-11-16T22:21:17.789 INFO:teuthology.orchestra.run.smithi122.stdout:(20/133): libradosstriper1-16.0.0-7198.gb637909 3.6 MB/s | 480 kB 00:00 2020-11-16T22:21:17.813 INFO:teuthology.orchestra.run.smithi173.stdout:(21/133): libradosstriper1-16.0.0-7198.gb637909 3.2 MB/s | 480 kB 00:00 2020-11-16T22:21:17.890 INFO:teuthology.orchestra.run.smithi173.stdout:(22/133): python3-ceph-argparse-16.0.0-7198.gb6 517 kB/s | 40 kB 00:00 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi173.stdout:(23/133): python3-ceph-common-16.0.0-7198.gb637 961 kB/s | 73 kB 00:00 2020-11-16T22:21:18.048 INFO:teuthology.orchestra.run.smithi173.stdout:(24/133): python3-cephfs-16.0.0-7198.gb637909a. 2.4 MB/s | 204 kB 00:00 2020-11-16T22:21:18.278 INFO:teuthology.orchestra.run.smithi173.stdout:(25/133): python3-rados-16.0.0-7198.gb637909a.e 1.6 MB/s | 375 kB 00:00 2020-11-16T22:21:18.300 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2020-11-16T22:21:18.300 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2020-11-16T22:21:18.413 INFO:teuthology.orchestra.run.smithi173.stdout:(26/133): python3-rbd-16.0.0-7198.gb637909a.el8 2.6 MB/s | 353 kB 00:00 2020-11-16T22:21:18.478 INFO:teuthology.orchestra.run.smithi173.stdout:(27/133): python3-rgw-16.0.0-7198.gb637909a.el8 1.7 MB/s | 113 kB 00:00 2020-11-16T22:21:18.539 INFO:teuthology.orchestra.run.smithi173.stdout:(28/133): rbd-fuse-16.0.0-7198.gb637909a.el8.x8 1.4 MB/s | 87 kB 00:00 2020-11-16T22:21:18.593 INFO:teuthology.orchestra.run.smithi173.stdout:(29/133): librgw2-16.0.0-7198.gb637909a.el8.x86 4.3 MB/s | 3.7 MB 00:00 2020-11-16T22:21:18.613 INFO:teuthology.orchestra.run.smithi122.stdout:(21/133): ceph-common-16.0.0-7198.gb637909a.el8 1.8 MB/s | 23 MB 00:12 2020-11-16T22:21:18.637 INFO:teuthology.orchestra.run.smithi173.stdout:(30/133): rbd-nbd-16.0.0-7198.gb637909a.el8.x86 3.8 MB/s | 168 kB 00:00 2020-11-16T22:21:18.675 INFO:teuthology.orchestra.run.smithi122.stdout:(22/133): python3-ceph-argparse-16.0.0-7198.gb6 640 kB/s | 40 kB 00:00 2020-11-16T22:21:18.694 INFO:teuthology.orchestra.run.smithi173.stdout:(31/133): ceph-grafana-dashboards-16.0.0-7198.g 424 kB/s | 24 kB 00:00 2020-11-16T22:21:18.727 INFO:teuthology.orchestra.run.smithi122.stdout:(23/133): librgw2-16.0.0-7198.gb637909a.el8.x86 3.9 MB/s | 3.7 MB 00:00 2020-11-16T22:21:18.731 INFO:teuthology.orchestra.run.smithi122.stdout:(24/133): python3-ceph-common-16.0.0-7198.gb637 1.3 MB/s | 73 kB 00:00 2020-11-16T22:21:18.751 INFO:teuthology.orchestra.run.smithi173.stdout:(32/133): ceph-mgr-cephadm-16.0.0-7198.gb637909 1.1 MB/s | 66 kB 00:00 2020-11-16T22:21:18.784 INFO:teuthology.orchestra.run.smithi122.stdout:(25/133): python3-cephfs-16.0.0-7198.gb637909a. 3.6 MB/s | 204 kB 00:00 2020-11-16T22:21:18.917 INFO:teuthology.orchestra.run.smithi122.stdout:(26/133): python3-rbd-16.0.0-7198.gb637909a.el8 2.6 MB/s | 353 kB 00:00 2020-11-16T22:21:18.918 INFO:teuthology.orchestra.run.smithi122.stdout:(27/133): python3-rados-16.0.0-7198.gb637909a.e 2.0 MB/s | 375 kB 00:00 2020-11-16T22:21:18.967 INFO:teuthology.orchestra.run.smithi194.stdout:(21/133): librgw2-16.0.0-7198.gb637909a.el8.x86 2.6 MB/s | 3.7 MB 00:01 2020-11-16T22:21:18.975 INFO:teuthology.orchestra.run.smithi122.stdout:(28/133): python3-rgw-16.0.0-7198.gb637909a.el8 1.9 MB/s | 113 kB 00:00 2020-11-16T22:21:18.978 INFO:teuthology.orchestra.run.smithi122.stdout:(29/133): rbd-fuse-16.0.0-7198.gb637909a.el8.x8 1.4 MB/s | 87 kB 00:00 2020-11-16T22:21:19.047 INFO:teuthology.orchestra.run.smithi122.stdout:(30/133): rbd-nbd-16.0.0-7198.gb637909a.el8.x86 2.4 MB/s | 168 kB 00:00 2020-11-16T22:21:19.106 INFO:teuthology.orchestra.run.smithi122.stdout:(31/133): ceph-grafana-dashboards-16.0.0-7198.g 411 kB/s | 24 kB 00:00 2020-11-16T22:21:19.129 INFO:teuthology.orchestra.run.smithi194.stdout:(22/133): ceph-osd-16.0.0-7198.gb637909a.el8.x8 1.8 MB/s | 18 MB 00:09 2020-11-16T22:21:19.130 INFO:teuthology.orchestra.run.smithi194.stdout:(23/133): python3-ceph-argparse-16.0.0-7198.gb6 244 kB/s | 40 kB 00:00 2020-11-16T22:21:19.167 INFO:teuthology.orchestra.run.smithi122.stdout:(32/133): ceph-mgr-cephadm-16.0.0-7198.gb637909 1.1 MB/s | 66 kB 00:00 2020-11-16T22:21:19.193 INFO:teuthology.orchestra.run.smithi194.stdout:(24/133): python3-ceph-common-16.0.0-7198.gb637 1.1 MB/s | 73 kB 00:00 2020-11-16T22:21:19.271 INFO:teuthology.orchestra.run.smithi194.stdout:(25/133): python3-cephfs-16.0.0-7198.gb637909a. 1.4 MB/s | 204 kB 00:00 2020-11-16T22:21:19.417 INFO:teuthology.orchestra.run.smithi194.stdout:(26/133): python3-rados-16.0.0-7198.gb637909a.e 1.6 MB/s | 375 kB 00:00 2020-11-16T22:21:19.482 INFO:teuthology.orchestra.run.smithi194.stdout:(27/133): python3-rbd-16.0.0-7198.gb637909a.el8 1.6 MB/s | 353 kB 00:00 2020-11-16T22:21:19.485 INFO:teuthology.orchestra.run.smithi173.stdout:(33/133): ceph-mgr-dashboard-16.0.0-7198.gb6379 4.3 MB/s | 3.2 MB 00:00 2020-11-16T22:21:19.485 INFO:teuthology.orchestra.run.smithi194.stdout:(28/133): python3-rgw-16.0.0-7198.gb637909a.el8 1.7 MB/s | 113 kB 00:00 2020-11-16T22:21:19.537 INFO:teuthology.orchestra.run.smithi194.stdout:(29/133): rbd-fuse-16.0.0-7198.gb637909a.el8.x8 1.6 MB/s | 87 kB 00:00 2020-11-16T22:21:19.542 INFO:teuthology.orchestra.run.smithi173.stdout:(34/133): rbd-mirror-16.0.0-7198.gb637909a.el8. 3.1 MB/s | 3.1 MB 00:01 2020-11-16T22:21:19.620 INFO:teuthology.orchestra.run.smithi173.stdout:(35/133): ceph-mgr-modules-core-16.0.0-7198.gb6 2.5 MB/s | 198 kB 00:00 2020-11-16T22:21:19.654 INFO:teuthology.orchestra.run.smithi122.stdout:(33/133): rbd-mirror-16.0.0-7198.gb637909a.el8. 4.6 MB/s | 3.1 MB 00:00 2020-11-16T22:21:19.670 INFO:teuthology.orchestra.run.smithi194.stdout:(30/133): rbd-nbd-16.0.0-7198.gb637909a.el8.x86 1.2 MB/s | 168 kB 00:00 2020-11-16T22:21:19.678 INFO:teuthology.orchestra.run.smithi173.stdout:(36/133): ceph-mgr-rook-16.0.0-7198.gb637909a.e 475 kB/s | 27 kB 00:00 2020-11-16T22:21:19.738 INFO:teuthology.orchestra.run.smithi194.stdout:(31/133): ceph-grafana-dashboards-16.0.0-7198.g 352 kB/s | 24 kB 00:00 2020-11-16T22:21:19.746 INFO:teuthology.orchestra.run.smithi173.stdout:(37/133): ceph-prometheus-alerts-16.0.0-7198.gb 130 kB/s | 8.8 kB 00:00 2020-11-16T22:21:19.783 INFO:teuthology.orchestra.run.smithi173.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2020-11-16T22:21:19.791 INFO:teuthology.orchestra.run.smithi173.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 79 MB/s | 688 kB 00:00 2020-11-16T22:21:19.796 INFO:teuthology.orchestra.run.smithi173.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 71 MB/s | 320 kB 00:00 2020-11-16T22:21:19.798 INFO:teuthology.orchestra.run.smithi173.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 62 MB/s | 161 kB 00:00 2020-11-16T22:21:19.803 INFO:teuthology.orchestra.run.smithi173.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 74 MB/s | 353 kB 00:00 2020-11-16T22:21:19.809 INFO:teuthology.orchestra.run.smithi194.stdout:(32/133): ceph-mgr-cephadm-16.0.0-7198.gb637909 944 kB/s | 66 kB 00:00 2020-11-16T22:21:19.812 INFO:teuthology.orchestra.run.smithi173.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 79 MB/s | 709 kB 00:00 2020-11-16T22:21:19.816 INFO:teuthology.orchestra.run.smithi173.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 68 MB/s | 259 kB 00:00 2020-11-16T22:21:19.819 INFO:teuthology.orchestra.run.smithi173.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 64 MB/s | 188 kB 00:00 2020-11-16T22:21:19.870 INFO:teuthology.orchestra.run.smithi173.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 86 MB/s | 4.3 MB 00:00 2020-11-16T22:21:19.921 INFO:teuthology.orchestra.run.smithi173.stdout:(47/133): openblas-threads-0.3.3-2.el8.x86_64.r 86 MB/s | 4.4 MB 00:00 2020-11-16T22:21:19.981 INFO:teuthology.orchestra.run.smithi173.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 86 MB/s | 4.8 MB 00:00 2020-11-16T22:21:19.981 INFO:teuthology.orchestra.run.smithi173.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 61 MB/s | 214 kB 00:00 2020-11-16T22:21:19.988 INFO:teuthology.orchestra.run.smithi173.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 77 MB/s | 538 kB 00:00 2020-11-16T22:21:19.989 INFO:teuthology.orchestra.run.smithi173.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 26 MB/s | 27 kB 00:00 2020-11-16T22:21:19.990 INFO:teuthology.orchestra.run.smithi173.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 17 MB/s | 20 kB 00:00 2020-11-16T22:21:20.009 INFO:teuthology.orchestra.run.smithi173.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 83 MB/s | 1.5 MB 00:00 2020-11-16T22:21:20.011 INFO:teuthology.orchestra.run.smithi173.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 59 MB/s | 157 kB 00:00 2020-11-16T22:21:20.012 INFO:teuthology.orchestra.run.smithi173.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 29 MB/s | 39 kB 00:00 2020-11-16T22:21:20.056 INFO:teuthology.orchestra.run.smithi173.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 86 MB/s | 3.7 MB 00:00 2020-11-16T22:21:20.060 INFO:teuthology.orchestra.run.smithi173.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 62 MB/s | 225 kB 00:00 2020-11-16T22:21:20.061 INFO:teuthology.orchestra.run.smithi173.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 33 MB/s | 44 kB 00:00 2020-11-16T22:21:20.064 INFO:teuthology.orchestra.run.smithi173.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 55 MB/s | 126 kB 00:00 2020-11-16T22:21:20.066 INFO:teuthology.orchestra.run.smithi173.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 50 MB/s | 110 kB 00:00 2020-11-16T22:21:20.068 INFO:teuthology.orchestra.run.smithi173.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 33 MB/s | 54 kB 00:00 2020-11-16T22:21:20.272 INFO:teuthology.orchestra.run.smithi173.stdout:(62/133): python3-scipy-1.0.0-20.module_el8.1.0 71 MB/s | 15 MB 00:00 2020-11-16T22:21:20.273 INFO:teuthology.orchestra.run.smithi173.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 18 MB/s | 27 kB 00:00 2020-11-16T22:21:20.279 INFO:teuthology.orchestra.run.smithi173.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 74 MB/s | 457 kB 00:00 2020-11-16T22:21:20.284 INFO:teuthology.orchestra.run.smithi173.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 69 MB/s | 298 kB 00:00 2020-11-16T22:21:20.285 INFO:teuthology.orchestra.run.smithi173.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 34 MB/s | 54 kB 00:00 2020-11-16T22:21:20.287 INFO:teuthology.orchestra.run.smithi173.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 48 MB/s | 82 kB 00:00 2020-11-16T22:21:20.289 INFO:teuthology.orchestra.run.smithi173.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 17 MB/s | 21 kB 00:00 2020-11-16T22:21:20.290 INFO:teuthology.orchestra.run.smithi173.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 30 MB/s | 48 kB 00:00 2020-11-16T22:21:20.297 INFO:teuthology.orchestra.run.smithi173.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 77 MB/s | 548 kB 00:00 2020-11-16T22:21:20.321 INFO:teuthology.orchestra.run.smithi173.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 796 kB/s | 19 kB 00:00 2020-11-16T22:21:20.323 INFO:teuthology.orchestra.run.smithi173.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 27 MB/s | 38 kB 00:00 2020-11-16T22:21:20.327 INFO:teuthology.orchestra.run.smithi122.stdout:(34/133): ceph-test-16.0.0-7198.gb637909a.el8.x 5.8 MB/s | 47 MB 00:08 2020-11-16T22:21:20.331 INFO:teuthology.orchestra.run.smithi173.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 78 MB/s | 638 kB 00:00 2020-11-16T22:21:20.334 INFO:teuthology.orchestra.run.smithi173.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 56 MB/s | 169 kB 00:00 2020-11-16T22:21:20.336 INFO:teuthology.orchestra.run.smithi173.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 32 MB/s | 47 kB 00:00 2020-11-16T22:21:20.339 INFO:teuthology.orchestra.run.smithi173.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 64 MB/s | 199 kB 00:00 2020-11-16T22:21:20.344 INFO:teuthology.orchestra.run.smithi173.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 16 MB/s | 86 kB 00:00 2020-11-16T22:21:20.346 INFO:teuthology.orchestra.run.smithi173.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 48 MB/s | 95 kB 00:00 2020-11-16T22:21:20.348 INFO:teuthology.orchestra.run.smithi173.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 15 MB/s | 18 kB 00:00 2020-11-16T22:21:20.351 INFO:teuthology.orchestra.run.smithi173.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 67 MB/s | 249 kB 00:00 2020-11-16T22:21:20.379 INFO:teuthology.orchestra.run.smithi173.stdout:(81/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 84 MB/s | 2.3 MB 00:00 2020-11-16T22:21:20.386 INFO:teuthology.orchestra.run.smithi173.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 77 MB/s | 605 kB 00:00 2020-11-16T22:21:20.390 INFO:teuthology.orchestra.run.smithi173.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 65 MB/s | 228 kB 00:00 2020-11-16T22:21:20.393 INFO:teuthology.orchestra.run.smithi173.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 67 MB/s | 220 kB 00:00 2020-11-16T22:21:20.397 INFO:teuthology.orchestra.run.smithi173.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 60 MB/s | 195 kB 00:00 2020-11-16T22:21:20.398 INFO:teuthology.orchestra.run.smithi173.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 35 MB/s | 43 kB 00:00 2020-11-16T22:21:20.400 INFO:teuthology.orchestra.run.smithi122.stdout:(35/133): ceph-mgr-modules-core-16.0.0-7198.gb6 2.7 MB/s | 198 kB 00:00 2020-11-16T22:21:20.401 INFO:teuthology.orchestra.run.smithi173.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 59 MB/s | 155 kB 00:00 2020-11-16T22:21:20.402 INFO:teuthology.orchestra.run.smithi173.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 25 MB/s | 34 kB 00:00 2020-11-16T22:21:20.405 INFO:teuthology.orchestra.run.smithi173.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 54 MB/s | 123 kB 00:00 2020-11-16T22:21:20.406 INFO:teuthology.orchestra.run.smithi173.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 36 MB/s | 43 kB 00:00 2020-11-16T22:21:20.409 INFO:teuthology.orchestra.run.smithi173.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 54 MB/s | 176 kB 00:00 2020-11-16T22:21:20.411 INFO:teuthology.orchestra.run.smithi173.stdout:(92/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 41 MB/s | 69 kB 00:00 2020-11-16T22:21:20.412 INFO:teuthology.orchestra.run.smithi173.stdout:(93/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 37 MB/s | 56 kB 00:00 2020-11-16T22:21:20.466 INFO:teuthology.orchestra.run.smithi122.stdout:(36/133): ceph-mgr-rook-16.0.0-7198.gb637909a.e 416 kB/s | 27 kB 00:00 2020-11-16T22:21:20.494 INFO:teuthology.orchestra.run.smithi173.stdout:(94/133): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2020-11-16T22:21:20.516 INFO:teuthology.orchestra.run.smithi122.stdout:(37/133): ceph-mgr-dashboard-16.0.0-7198.gb6379 2.3 MB/s | 3.2 MB 00:01 2020-11-16T22:21:20.525 INFO:teuthology.orchestra.run.smithi122.stdout:(38/133): ceph-prometheus-alerts-16.0.0-7198.gb 150 kB/s | 8.8 kB 00:00 2020-11-16T22:21:20.531 INFO:teuthology.orchestra.run.smithi173.stdout:(95/133): gperftools-libs-2.7-6.el8.x86_64.rpm 8.4 MB/s | 315 kB 00:00 2020-11-16T22:21:20.549 INFO:teuthology.orchestra.run.smithi173.stdout:(96/133): leveldb-1.22-1.el8.x86_64.rpm 10 MB/s | 181 kB 00:00 2020-11-16T22:21:20.553 INFO:teuthology.orchestra.run.smithi122.stdout:(39/133): automake-1.16.1-6.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2020-11-16T22:21:20.559 INFO:teuthology.orchestra.run.smithi122.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 61 MB/s | 320 kB 00:00 2020-11-16T22:21:20.562 INFO:teuthology.orchestra.run.smithi122.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 45 MB/s | 161 kB 00:00 2020-11-16T22:21:20.565 INFO:teuthology.orchestra.run.smithi173.stdout:(97/133): liboath-2.6.2-3.el8.x86_64.rpm 3.5 MB/s | 59 kB 00:00 2020-11-16T22:21:20.569 INFO:teuthology.orchestra.run.smithi122.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 56 MB/s | 353 kB 00:00 2020-11-16T22:21:20.577 INFO:teuthology.orchestra.run.smithi122.stdout:(43/133): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2020-11-16T22:21:20.581 INFO:teuthology.orchestra.run.smithi122.stdout:(44/133): libtool-2.4.6-25.el8.x86_64.rpm 57 MB/s | 709 kB 00:00 2020-11-16T22:21:20.582 INFO:teuthology.orchestra.run.smithi173.stdout:(98/133): libunwind-1.3.1-3.el8.x86_64.rpm 4.4 MB/s | 75 kB 00:00 2020-11-16T22:21:20.585 INFO:teuthology.orchestra.run.smithi122.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 50 MB/s | 188 kB 00:00 2020-11-16T22:21:20.587 INFO:teuthology.orchestra.run.smithi122.stdout:(46/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 27 MB/s | 259 kB 00:00 2020-11-16T22:21:20.598 INFO:teuthology.orchestra.run.smithi173.stdout:(99/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-11-16T22:21:20.617 INFO:teuthology.orchestra.run.smithi173.stdout:(100/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.4 MB/s | 44 kB 00:00 2020-11-16T22:21:20.634 INFO:teuthology.orchestra.run.smithi173.stdout:(101/133): python3-beautifulsoup4-4.6.3-2.el8.1 10 MB/s | 185 kB 00:00 2020-11-16T22:21:20.649 INFO:teuthology.orchestra.run.smithi122.stdout:(47/133): openblas-0.3.3-2.el8.x86_64.rpm 68 MB/s | 4.3 MB 00:00 2020-11-16T22:21:20.651 INFO:teuthology.orchestra.run.smithi173.stdout:(102/133): python3-cachetools-3.1.1-4.el8.noarc 1.9 MB/s | 33 kB 00:00 2020-11-16T22:21:20.667 INFO:teuthology.orchestra.run.smithi173.stdout:(103/133): python3-certifi-2018.10.15-7.el8.noa 1.0 MB/s | 16 kB 00:00 2020-11-16T22:21:20.685 INFO:teuthology.orchestra.run.smithi173.stdout:(104/133): python3-cheroot-8.2.1-1.el8.noarch.r 8.0 MB/s | 145 kB 00:00 2020-11-16T22:21:20.718 INFO:teuthology.orchestra.run.smithi173.stdout:(105/133): python3-cherrypy-18.4.0-1.el8.noarch 11 MB/s | 384 kB 00:00 2020-11-16T22:21:20.721 INFO:teuthology.orchestra.run.smithi122.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 67 MB/s | 4.8 MB 00:00 2020-11-16T22:21:20.726 INFO:teuthology.orchestra.run.smithi122.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 37 MB/s | 214 kB 00:00 2020-11-16T22:21:20.735 INFO:teuthology.orchestra.run.smithi173.stdout:(106/133): python3-cssselect-0.9.2-10.el8.noarc 2.4 MB/s | 40 kB 00:00 2020-11-16T22:21:20.738 INFO:teuthology.orchestra.run.smithi122.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 48 MB/s | 538 kB 00:00 2020-11-16T22:21:20.750 INFO:teuthology.orchestra.run.smithi122.stdout:(51/133): openblas-threads-0.3.3-2.el8.x86_64.r 27 MB/s | 4.4 MB 00:00 2020-11-16T22:21:20.751 INFO:teuthology.orchestra.run.smithi122.stdout:(52/133): python3-jsonpatch-1.21-2.el8.noarch.r 2.1 MB/s | 27 kB 00:00 2020-11-16T22:21:20.751 INFO:teuthology.orchestra.run.smithi122.stdout:(53/133): python3-jsonpointer-1.10-11.el8.noarc 17 MB/s | 20 kB 00:00 2020-11-16T22:21:20.754 INFO:teuthology.orchestra.run.smithi173.stdout:(107/133): python3-execnet-1.7.1-1.el8.noarch.r 18 MB/s | 355 kB 00:00 2020-11-16T22:21:20.756 INFO:teuthology.orchestra.run.smithi122.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 36 MB/s | 157 kB 00:00 2020-11-16T22:21:20.760 INFO:teuthology.orchestra.run.smithi122.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 9.5 MB/s | 39 kB 00:00 2020-11-16T22:21:20.771 INFO:teuthology.orchestra.run.smithi173.stdout:(108/133): python3-google-auth-1.1.1-10.el8.noa 4.7 MB/s | 81 kB 00:00 2020-11-16T22:21:20.773 INFO:teuthology.orchestra.run.smithi122.stdout:(56/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 66 MB/s | 1.5 MB 00:00 2020-11-16T22:21:20.780 INFO:teuthology.orchestra.run.smithi122.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 36 MB/s | 225 kB 00:00 2020-11-16T22:21:20.783 INFO:teuthology.orchestra.run.smithi122.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 13 MB/s | 44 kB 00:00 2020-11-16T22:21:20.787 INFO:teuthology.orchestra.run.smithi173.stdout:(109/133): python3-jaraco-6.2-6.el8.noarch.rpm 686 kB/s | 11 kB 00:00 2020-11-16T22:21:20.789 INFO:teuthology.orchestra.run.smithi122.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 24 MB/s | 126 kB 00:00 2020-11-16T22:21:20.793 INFO:teuthology.orchestra.run.smithi122.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 26 MB/s | 110 kB 00:00 2020-11-16T22:21:20.797 INFO:teuthology.orchestra.run.smithi122.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 15 MB/s | 54 kB 00:00 2020-11-16T22:21:20.804 INFO:teuthology.orchestra.run.smithi173.stdout:(110/133): python3-jaraco-functools-2.0-4.el8.n 1.0 MB/s | 18 kB 00:00 2020-11-16T22:21:20.826 INFO:teuthology.orchestra.run.smithi122.stdout:(62/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 56 MB/s | 3.7 MB 00:00 2020-11-16T22:21:20.830 INFO:teuthology.orchestra.run.smithi122.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 7.3 MB/s | 27 kB 00:00 2020-11-16T22:21:20.842 INFO:teuthology.orchestra.run.smithi122.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 39 MB/s | 457 kB 00:00 2020-11-16T22:21:20.851 INFO:teuthology.orchestra.run.smithi122.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 32 MB/s | 298 kB 00:00 2020-11-16T22:21:20.856 INFO:teuthology.orchestra.run.smithi122.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 13 MB/s | 54 kB 00:00 2020-11-16T22:21:20.860 INFO:teuthology.orchestra.run.smithi122.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 18 MB/s | 82 kB 00:00 2020-11-16T22:21:20.864 INFO:teuthology.orchestra.run.smithi122.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 5.7 MB/s | 21 kB 00:00 2020-11-16T22:21:20.870 INFO:teuthology.orchestra.run.smithi173.stdout:(111/133): python3-kubernetes-11.0.0-2.el8.noar 18 MB/s | 1.2 MB 00:00 2020-11-16T22:21:20.879 INFO:teuthology.orchestra.run.smithi122.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2020-11-16T22:21:20.887 INFO:teuthology.orchestra.run.smithi173.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.9 MB/s | 49 kB 00:00 2020-11-16T22:21:20.893 INFO:teuthology.orchestra.run.smithi122.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 37 MB/s | 548 kB 00:00 2020-11-16T22:21:20.897 INFO:teuthology.orchestra.run.smithi194.stdout:(33/133): rbd-mirror-16.0.0-7198.gb637909a.el8. 2.2 MB/s | 3.1 MB 00:01 2020-11-16T22:21:20.898 INFO:teuthology.orchestra.run.smithi122.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 4.3 MB/s | 19 kB 00:00 2020-11-16T22:21:20.904 INFO:teuthology.orchestra.run.smithi173.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 3.5 MB/s | 59 kB 00:00 2020-11-16T22:21:20.913 INFO:teuthology.orchestra.run.smithi122.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 2.4 MB/s | 38 kB 00:00 2020-11-16T22:21:20.922 INFO:teuthology.orchestra.run.smithi173.stdout:(114/133): python3-pecan-1.3.2-9.el8.noarch.rpm 15 MB/s | 283 kB 00:00 2020-11-16T22:21:20.928 INFO:teuthology.orchestra.run.smithi122.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 42 MB/s | 638 kB 00:00 2020-11-16T22:21:20.934 INFO:teuthology.orchestra.run.smithi122.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 28 MB/s | 169 kB 00:00 2020-11-16T22:21:20.939 INFO:teuthology.orchestra.run.smithi173.stdout:(115/133): python3-portend-2.6-1.el8.noarch.rpm 963 kB/s | 16 kB 00:00 2020-11-16T22:21:20.939 INFO:teuthology.orchestra.run.smithi122.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 9.5 MB/s | 47 kB 00:00 2020-11-16T22:21:20.946 INFO:teuthology.orchestra.run.smithi122.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 31 MB/s | 199 kB 00:00 2020-11-16T22:21:20.951 INFO:teuthology.orchestra.run.smithi122.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 15 MB/s | 86 kB 00:00 2020-11-16T22:21:20.955 INFO:teuthology.orchestra.run.smithi173.stdout:(116/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.8 MB/s | 48 kB 00:00 2020-11-16T22:21:20.957 INFO:teuthology.orchestra.run.smithi122.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 18 MB/s | 95 kB 00:00 2020-11-16T22:21:20.963 INFO:teuthology.orchestra.run.smithi122.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 3.1 MB/s | 18 kB 00:00 2020-11-16T22:21:20.970 INFO:teuthology.orchestra.run.smithi122.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 33 MB/s | 249 kB 00:00 2020-11-16T22:21:20.972 INFO:teuthology.orchestra.run.smithi173.stdout:(117/133): python3-repoze-lru-0.7-6.el8.noarch. 2.0 MB/s | 34 kB 00:00 2020-11-16T22:21:20.990 INFO:teuthology.orchestra.run.smithi173.stdout:(118/133): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-11-16T22:21:21.007 INFO:teuthology.orchestra.run.smithi173.stdout:(119/133): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-11-16T22:21:21.023 INFO:teuthology.orchestra.run.smithi173.stdout:(120/133): python3-simplegeneric-0.8.1-17.el8.n 1.1 MB/s | 19 kB 00:00 2020-11-16T22:21:21.035 INFO:teuthology.orchestra.run.smithi194.stdout:(34/133): ceph-mgr-dashboard-16.0.0-7198.gb6379 2.6 MB/s | 3.2 MB 00:01 2020-11-16T22:21:21.047 INFO:teuthology.orchestra.run.smithi173.stdout:(121/133): ceph-mgr-diskprediction-local-16.0.0 5.5 MB/s | 8.6 MB 00:01 2020-11-16T22:21:21.086 INFO:teuthology.orchestra.run.smithi194.stdout:(35/133): ceph-mgr-modules-core-16.0.0-7198.gb6 3.8 MB/s | 198 kB 00:00 2020-11-16T22:21:21.100 INFO:teuthology.orchestra.run.smithi173.stdout:(122/133): python3-singledispatch-3.4.0.3-18.el 314 kB/s | 24 kB 00:00 2020-11-16T22:21:21.101 INFO:teuthology.orchestra.run.smithi122.stdout:(81/133): python3-scipy-1.0.0-20.module_el8.1.0 48 MB/s | 15 MB 00:00 2020-11-16T22:21:21.111 INFO:teuthology.orchestra.run.smithi194.stdout:(36/133): ceph-mgr-rook-16.0.0-7198.gb637909a.e 1.1 MB/s | 27 kB 00:00 2020-11-16T22:21:21.117 INFO:teuthology.orchestra.run.smithi173.stdout:(123/133): python3-trustme-0.6.0-4.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2020-11-16T22:21:21.124 INFO:teuthology.orchestra.run.smithi122.stdout:(82/133): ceph-mgr-diskprediction-local-16.0.0- 5.8 MB/s | 8.6 MB 00:01 2020-11-16T22:21:21.133 INFO:teuthology.orchestra.run.smithi122.stdout:(83/133): pcre2-devel-10.32-1.el8.x86_64.rpm 19 MB/s | 605 kB 00:00 2020-11-16T22:21:21.135 INFO:teuthology.orchestra.run.smithi173.stdout:(124/133): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-11-16T22:21:21.137 INFO:teuthology.orchestra.run.smithi194.stdout:(37/133): ceph-prometheus-alerts-16.0.0-7198.gb 350 kB/s | 8.8 kB 00:00 2020-11-16T22:21:21.138 INFO:teuthology.orchestra.run.smithi122.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 42 MB/s | 220 kB 00:00 2020-11-16T22:21:21.139 INFO:teuthology.orchestra.run.smithi173.stdout:(125/133): python3-tempora-1.14.1-5.el8.noarch. 320 kB/s | 29 kB 00:00 2020-11-16T22:21:21.145 INFO:teuthology.orchestra.run.smithi122.stdout:(85/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 13 MB/s | 2.3 MB 00:00 2020-11-16T22:21:21.147 INFO:teuthology.orchestra.run.smithi122.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 20 MB/s | 43 kB 00:00 2020-11-16T22:21:21.148 INFO:teuthology.orchestra.run.smithi122.stdout:(87/133): python3-chardet-3.0.4-7.el8.noarch.rp 20 MB/s | 195 kB 00:00 2020-11-16T22:21:21.153 INFO:teuthology.orchestra.run.smithi173.stdout:(126/133): python3-webob-1.8.5-1.el8.1.noarch.r 14 MB/s | 251 kB 00:00 2020-11-16T22:21:21.170 INFO:teuthology.orchestra.run.smithi173.stdout:(127/133): python3-webtest-2.0.33-1.el8.noarch. 4.9 MB/s | 85 kB 00:00 2020-11-16T22:21:21.175 INFO:teuthology.orchestra.run.smithi194.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2020-11-16T22:21:21.178 INFO:teuthology.orchestra.run.smithi173.stdout:(128/133): python3-websocket-client-0.56.0-5.el 1.5 MB/s | 61 kB 00:00 2020-11-16T22:21:21.183 INFO:teuthology.orchestra.run.smithi194.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 77 MB/s | 688 kB 00:00 2020-11-16T22:21:21.187 INFO:teuthology.orchestra.run.smithi173.stdout:(129/133): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-11-16T22:21:21.188 INFO:teuthology.orchestra.run.smithi194.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 70 MB/s | 320 kB 00:00 2020-11-16T22:21:21.191 INFO:teuthology.orchestra.run.smithi194.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 57 MB/s | 161 kB 00:00 2020-11-16T22:21:21.196 INFO:teuthology.orchestra.run.smithi194.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 72 MB/s | 353 kB 00:00 2020-11-16T22:21:21.198 INFO:teuthology.orchestra.run.smithi173.stdout:(130/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.3 MB/s | 69 kB 00:00 2020-11-16T22:21:21.205 INFO:teuthology.orchestra.run.smithi194.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 78 MB/s | 709 kB 00:00 2020-11-16T22:21:21.209 INFO:teuthology.orchestra.run.smithi194.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 65 MB/s | 259 kB 00:00 2020-11-16T22:21:21.212 INFO:teuthology.orchestra.run.smithi122.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 536 kB/s | 34 kB 00:00 2020-11-16T22:21:21.212 INFO:teuthology.orchestra.run.smithi194.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 60 MB/s | 188 kB 00:00 2020-11-16T22:21:21.214 INFO:teuthology.orchestra.run.smithi122.stdout:(89/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2020-11-16T22:21:21.216 INFO:teuthology.orchestra.run.smithi122.stdout:(90/133): python3-requests-2.20.0-2.1.el8_1.noa 38 MB/s | 123 kB 00:00 2020-11-16T22:21:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 12 MB/s | 43 kB 00:00 2020-11-16T22:21:21.221 INFO:teuthology.orchestra.run.smithi122.stdout:(92/133): python3-urllib3-1.24.2-2.el8.noarch.r 35 MB/s | 176 kB 00:00 2020-11-16T22:21:21.222 INFO:teuthology.orchestra.run.smithi122.stdout:(93/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 16 MB/s | 69 kB 00:00 2020-11-16T22:21:21.224 INFO:teuthology.orchestra.run.smithi122.stdout:(94/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 20 MB/s | 56 kB 00:00 2020-11-16T22:21:21.231 INFO:teuthology.orchestra.run.smithi122.stdout:(95/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 2.1 MB/s | 228 kB 00:00 2020-11-16T22:21:21.269 INFO:teuthology.orchestra.run.smithi194.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 76 MB/s | 4.3 MB 00:00 2020-11-16T22:21:21.304 INFO:teuthology.orchestra.run.smithi122.stdout:(96/133): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2020-11-16T22:21:21.312 INFO:teuthology.orchestra.run.smithi122.stdout:(97/133): leveldb-1.22-1.el8.x86_64.rpm 2.2 MB/s | 181 kB 00:00 2020-11-16T22:21:21.321 INFO:teuthology.orchestra.run.smithi122.stdout:(98/133): liboath-2.6.2-3.el8.x86_64.rpm 3.4 MB/s | 59 kB 00:00 2020-11-16T22:21:21.321 INFO:teuthology.orchestra.run.smithi194.stdout:(47/133): openblas-threads-0.3.3-2.el8.x86_64.r 85 MB/s | 4.4 MB 00:00 2020-11-16T22:21:21.322 INFO:teuthology.orchestra.run.smithi122.stdout:(99/133): gperftools-libs-2.7-6.el8.x86_64.rpm 3.2 MB/s | 315 kB 00:00 2020-11-16T22:21:21.328 INFO:teuthology.orchestra.run.smithi122.stdout:(100/133): libunwind-1.3.1-3.el8.x86_64.rpm 4.4 MB/s | 75 kB 00:00 2020-11-16T22:21:21.337 INFO:teuthology.orchestra.run.smithi122.stdout:(101/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-11-16T22:21:21.339 INFO:teuthology.orchestra.run.smithi122.stdout:(102/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.6 MB/s | 44 kB 00:00 2020-11-16T22:21:21.354 INFO:teuthology.orchestra.run.smithi122.stdout:(103/133): python3-cachetools-3.1.1-4.el8.noarc 1.9 MB/s | 33 kB 00:00 2020-11-16T22:21:21.355 INFO:teuthology.orchestra.run.smithi122.stdout:(104/133): python3-certifi-2018.10.15-7.el8.noa 1.0 MB/s | 16 kB 00:00 2020-11-16T22:21:21.361 INFO:teuthology.orchestra.run.smithi122.stdout:(105/133): python3-beautifulsoup4-4.6.3-2.el8.1 5.6 MB/s | 185 kB 00:00 2020-11-16T22:21:21.376 INFO:teuthology.orchestra.run.smithi194.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 86 MB/s | 4.8 MB 00:00 2020-11-16T22:21:21.378 INFO:teuthology.orchestra.run.smithi122.stdout:(106/133): python3-cssselect-0.9.2-10.el8.noarc 2.4 MB/s | 40 kB 00:00 2020-11-16T22:21:21.380 INFO:teuthology.orchestra.run.smithi194.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 61 MB/s | 214 kB 00:00 2020-11-16T22:21:21.386 INFO:teuthology.orchestra.run.smithi122.stdout:(107/133): python3-cheroot-8.2.1-1.el8.noarch.r 4.4 MB/s | 145 kB 00:00 2020-11-16T22:21:21.388 INFO:teuthology.orchestra.run.smithi194.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 74 MB/s | 538 kB 00:00 2020-11-16T22:21:21.388 INFO:teuthology.orchestra.run.smithi122.stdout:(108/133): python3-cherrypy-18.4.0-1.el8.noarch 11 MB/s | 384 kB 00:00 2020-11-16T22:21:21.389 INFO:teuthology.orchestra.run.smithi194.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 22 MB/s | 27 kB 00:00 2020-11-16T22:21:21.390 INFO:teuthology.orchestra.run.smithi194.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 17 MB/s | 20 kB 00:00 2020-11-16T22:21:21.397 INFO:teuthology.orchestra.run.smithi122.stdout:(109/133): python3-execnet-1.7.1-1.el8.noarch.r 19 MB/s | 355 kB 00:00 2020-11-16T22:21:21.403 INFO:teuthology.orchestra.run.smithi122.stdout:(110/133): python3-google-auth-1.1.1-10.el8.noa 4.7 MB/s | 81 kB 00:00 2020-11-16T22:21:21.404 INFO:teuthology.orchestra.run.smithi122.stdout:(111/133): python3-jaraco-6.2-6.el8.noarch.rpm 685 kB/s | 11 kB 00:00 2020-11-16T22:21:21.409 INFO:teuthology.orchestra.run.smithi194.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 82 MB/s | 1.5 MB 00:00 2020-11-16T22:21:21.411 INFO:teuthology.orchestra.run.smithi194.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 60 MB/s | 157 kB 00:00 2020-11-16T22:21:21.413 INFO:teuthology.orchestra.run.smithi194.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 31 MB/s | 39 kB 00:00 2020-11-16T22:21:21.413 INFO:teuthology.orchestra.run.smithi122.stdout:(112/133): python3-jaraco-functools-2.0-4.el8.n 1.0 MB/s | 18 kB 00:00 2020-11-16T22:21:21.421 INFO:teuthology.orchestra.run.smithi122.stdout:(113/133): python3-logutils-0.3.5-11.el8.noarch 2.9 MB/s | 49 kB 00:00 2020-11-16T22:21:21.430 INFO:teuthology.orchestra.run.smithi122.stdout:(114/133): python3-more-itertools-7.2.0-3.el8.n 3.5 MB/s | 59 kB 00:00 2020-11-16T22:21:21.439 INFO:teuthology.orchestra.run.smithi122.stdout:(115/133): python3-pecan-1.3.2-9.el8.noarch.rpm 15 MB/s | 283 kB 00:00 2020-11-16T22:21:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 956 kB/s | 16 kB 00:00 2020-11-16T22:21:21.456 INFO:teuthology.orchestra.run.smithi122.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.8 MB/s | 48 kB 00:00 2020-11-16T22:21:21.457 INFO:teuthology.orchestra.run.smithi194.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 86 MB/s | 3.7 MB 00:00 2020-11-16T22:21:21.460 INFO:teuthology.orchestra.run.smithi194.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 61 MB/s | 225 kB 00:00 2020-11-16T22:21:21.462 INFO:teuthology.orchestra.run.smithi194.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 30 MB/s | 44 kB 00:00 2020-11-16T22:21:21.463 INFO:teuthology.orchestra.run.smithi122.stdout:(118/133): python3-repoze-lru-0.7-6.el8.noarch. 2.0 MB/s | 34 kB 00:00 2020-11-16T22:21:21.464 INFO:teuthology.orchestra.run.smithi194.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 53 MB/s | 126 kB 00:00 2020-11-16T22:21:21.467 INFO:teuthology.orchestra.run.smithi194.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 49 MB/s | 110 kB 00:00 2020-11-16T22:21:21.468 INFO:teuthology.orchestra.run.smithi194.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 36 MB/s | 54 kB 00:00 2020-11-16T22:21:21.470 INFO:teuthology.orchestra.run.smithi122.stdout:(119/133): python3-kubernetes-11.0.0-2.el8.noar 17 MB/s | 1.2 MB 00:00 2020-11-16T22:21:21.474 INFO:teuthology.orchestra.run.smithi122.stdout:(120/133): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-11-16T22:21:21.480 INFO:teuthology.orchestra.run.smithi122.stdout:(121/133): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi122.stdout:(122/133): python3-simplegeneric-0.8.1-17.el8.n 1.1 MB/s | 19 kB 00:00 2020-11-16T22:21:21.491 INFO:teuthology.orchestra.run.smithi122.stdout:(123/133): python3-singledispatch-3.4.0.3-18.el 1.4 MB/s | 24 kB 00:00 2020-11-16T22:21:21.496 INFO:teuthology.orchestra.run.smithi122.stdout:(124/133): python3-tempora-1.14.1-5.el8.noarch. 1.8 MB/s | 29 kB 00:00 2020-11-16T22:21:21.503 INFO:teuthology.orchestra.run.smithi122.stdout:(125/133): python3-trustme-0.6.0-4.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2020-11-16T22:21:21.509 INFO:teuthology.orchestra.run.smithi122.stdout:(126/133): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-11-16T22:21:21.514 INFO:teuthology.orchestra.run.smithi122.stdout:(127/133): python3-webob-1.8.5-1.el8.1.noarch.r 14 MB/s | 251 kB 00:00 2020-11-16T22:21:21.520 INFO:teuthology.orchestra.run.smithi122.stdout:(128/133): python3-websocket-client-0.56.0-5.el 3.6 MB/s | 61 kB 00:00 2020-11-16T22:21:21.526 INFO:teuthology.orchestra.run.smithi122.stdout:(129/133): python3-webtest-2.0.33-1.el8.noarch. 5.0 MB/s | 85 kB 00:00 2020-11-16T22:21:21.531 INFO:teuthology.orchestra.run.smithi122.stdout:(130/133): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-11-16T22:21:21.537 INFO:teuthology.orchestra.run.smithi122.stdout:(131/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 4.0 MB/s | 69 kB 00:00 2020-11-16T22:21:21.619 INFO:teuthology.orchestra.run.smithi173.stdout:(131/133): librados2-16.0.0-7198.gb637909a.el8. 8.5 MB/s | 3.7 MB 00:00 2020-11-16T22:21:21.684 INFO:teuthology.orchestra.run.smithi194.stdout:(62/133): python3-scipy-1.0.0-20.module_el8.1.0 67 MB/s | 15 MB 00:00 2020-11-16T22:21:21.686 INFO:teuthology.orchestra.run.smithi194.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 18 MB/s | 27 kB 00:00 2020-11-16T22:21:21.692 INFO:teuthology.orchestra.run.smithi194.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 72 MB/s | 457 kB 00:00 2020-11-16T22:21:21.697 INFO:teuthology.orchestra.run.smithi194.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 67 MB/s | 298 kB 00:00 2020-11-16T22:21:21.699 INFO:teuthology.orchestra.run.smithi194.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 34 MB/s | 54 kB 00:00 2020-11-16T22:21:21.701 INFO:teuthology.orchestra.run.smithi194.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 45 MB/s | 82 kB 00:00 2020-11-16T22:21:21.702 INFO:teuthology.orchestra.run.smithi194.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 19 MB/s | 21 kB 00:00 2020-11-16T22:21:21.704 INFO:teuthology.orchestra.run.smithi194.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 31 MB/s | 48 kB 00:00 2020-11-16T22:21:21.711 INFO:teuthology.orchestra.run.smithi194.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 76 MB/s | 548 kB 00:00 2020-11-16T22:21:21.712 INFO:teuthology.orchestra.run.smithi194.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 16 MB/s | 19 kB 00:00 2020-11-16T22:21:21.714 INFO:teuthology.orchestra.run.smithi194.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 29 MB/s | 38 kB 00:00 2020-11-16T22:21:21.722 INFO:teuthology.orchestra.run.smithi194.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 78 MB/s | 638 kB 00:00 2020-11-16T22:21:21.725 INFO:teuthology.orchestra.run.smithi194.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 60 MB/s | 169 kB 00:00 2020-11-16T22:21:21.726 INFO:teuthology.orchestra.run.smithi194.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 33 MB/s | 47 kB 00:00 2020-11-16T22:21:21.729 INFO:teuthology.orchestra.run.smithi194.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 60 MB/s | 199 kB 00:00 2020-11-16T22:21:21.732 INFO:teuthology.orchestra.run.smithi194.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 49 MB/s | 86 kB 00:00 2020-11-16T22:21:21.733 INFO:teuthology.orchestra.run.smithi194.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 49 MB/s | 95 kB 00:00 2020-11-16T22:21:21.734 INFO:teuthology.orchestra.run.smithi194.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 17 MB/s | 18 kB 00:00 2020-11-16T22:21:21.738 INFO:teuthology.orchestra.run.smithi194.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 67 MB/s | 249 kB 00:00 2020-11-16T22:21:21.766 INFO:teuthology.orchestra.run.smithi194.stdout:(81/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 84 MB/s | 2.3 MB 00:00 2020-11-16T22:21:21.773 INFO:teuthology.orchestra.run.smithi194.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 78 MB/s | 605 kB 00:00 2020-11-16T22:21:21.777 INFO:teuthology.orchestra.run.smithi194.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 63 MB/s | 228 kB 00:00 2020-11-16T22:21:21.780 INFO:teuthology.orchestra.run.smithi194.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 65 MB/s | 220 kB 00:00 2020-11-16T22:21:21.783 INFO:teuthology.orchestra.run.smithi194.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 66 MB/s | 195 kB 00:00 2020-11-16T22:21:21.785 INFO:teuthology.orchestra.run.smithi194.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 35 MB/s | 43 kB 00:00 2020-11-16T22:21:21.788 INFO:teuthology.orchestra.run.smithi194.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 58 MB/s | 155 kB 00:00 2020-11-16T22:21:21.789 INFO:teuthology.orchestra.run.smithi194.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 27 MB/s | 34 kB 00:00 2020-11-16T22:21:21.792 INFO:teuthology.orchestra.run.smithi194.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 56 MB/s | 123 kB 00:00 2020-11-16T22:21:21.793 INFO:teuthology.orchestra.run.smithi194.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 37 MB/s | 43 kB 00:00 2020-11-16T22:21:21.796 INFO:teuthology.orchestra.run.smithi194.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 66 MB/s | 176 kB 00:00 2020-11-16T22:21:21.797 INFO:teuthology.orchestra.run.smithi194.stdout:(92/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 44 MB/s | 69 kB 00:00 2020-11-16T22:21:21.799 INFO:teuthology.orchestra.run.smithi194.stdout:(93/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 38 MB/s | 56 kB 00:00 2020-11-16T22:21:21.841 INFO:teuthology.orchestra.run.smithi173.stdout:(132/133): librbd1-16.0.0-7198.gb637909a.el8.x8 3.7 MB/s | 2.4 MB 00:00 2020-11-16T22:21:21.897 INFO:teuthology.orchestra.run.smithi194.stdout:(94/133): fmt-6.2.1-1.el8.x86_64.rpm 1.2 MB/s | 116 kB 00:00 2020-11-16T22:21:21.937 INFO:teuthology.orchestra.run.smithi194.stdout:(95/133): gperftools-libs-2.7-6.el8.x86_64.rpm 7.7 MB/s | 315 kB 00:00 2020-11-16T22:21:21.958 INFO:teuthology.orchestra.run.smithi194.stdout:(96/133): leveldb-1.22-1.el8.x86_64.rpm 8.5 MB/s | 181 kB 00:00 2020-11-16T22:21:21.978 INFO:teuthology.orchestra.run.smithi194.stdout:(97/133): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2020-11-16T22:21:21.979 INFO:teuthology.orchestra.run.smithi122.stdout:(132/133): librados2-16.0.0-7198.gb637909a.el8. 8.1 MB/s | 3.7 MB 00:00 2020-11-16T22:21:21.998 INFO:teuthology.orchestra.run.smithi194.stdout:(98/133): libunwind-1.3.1-3.el8.x86_64.rpm 3.7 MB/s | 75 kB 00:00 2020-11-16T22:21:22.018 INFO:teuthology.orchestra.run.smithi194.stdout:(99/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.0 MB/s | 21 kB 00:00 2020-11-16T22:21:22.039 INFO:teuthology.orchestra.run.smithi194.stdout:(100/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.1 MB/s | 44 kB 00:00 2020-11-16T22:21:22.060 INFO:teuthology.orchestra.run.smithi194.stdout:(101/133): python3-beautifulsoup4-4.6.3-2.el8.1 8.7 MB/s | 185 kB 00:00 2020-11-16T22:21:22.080 INFO:teuthology.orchestra.run.smithi194.stdout:(102/133): python3-cachetools-3.1.1-4.el8.noarc 1.6 MB/s | 33 kB 00:00 2020-11-16T22:21:22.100 INFO:teuthology.orchestra.run.smithi194.stdout:(103/133): python3-certifi-2018.10.15-7.el8.noa 831 kB/s | 16 kB 00:00 2020-11-16T22:21:22.121 INFO:teuthology.orchestra.run.smithi194.stdout:(104/133): python3-cheroot-8.2.1-1.el8.noarch.r 6.8 MB/s | 145 kB 00:00 2020-11-16T22:21:22.161 INFO:teuthology.orchestra.run.smithi194.stdout:(105/133): python3-cherrypy-18.4.0-1.el8.noarch 9.5 MB/s | 384 kB 00:00 2020-11-16T22:21:22.180 INFO:teuthology.orchestra.run.smithi194.stdout:(106/133): python3-cssselect-0.9.2-10.el8.noarc 2.0 MB/s | 40 kB 00:00 2020-11-16T22:21:22.202 INFO:teuthology.orchestra.run.smithi194.stdout:(107/133): python3-execnet-1.7.1-1.el8.noarch.r 16 MB/s | 355 kB 00:00 2020-11-16T22:21:22.208 INFO:teuthology.orchestra.run.smithi122.stdout:(133/133): librbd1-16.0.0-7198.gb637909a.el8.x8 3.5 MB/s | 2.4 MB 00:00 2020-11-16T22:21:22.211 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2020-11-16T22:21:22.212 INFO:teuthology.orchestra.run.smithi122.stdout:Total 11 MB/s | 196 MB 00:17 2020-11-16T22:21:22.223 INFO:teuthology.orchestra.run.smithi194.stdout:(108/133): python3-google-auth-1.1.1-10.el8.noa 3.9 MB/s | 81 kB 00:00 2020-11-16T22:21:22.242 INFO:teuthology.orchestra.run.smithi194.stdout:(109/133): python3-jaraco-6.2-6.el8.noarch.rpm 570 kB/s | 11 kB 00:00 2020-11-16T22:21:22.262 INFO:teuthology.orchestra.run.smithi194.stdout:(110/133): python3-jaraco-functools-2.0-4.el8.n 892 kB/s | 18 kB 00:00 2020-11-16T22:21:22.268 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2020-11-16T22:21:22.342 INFO:teuthology.orchestra.run.smithi194.stdout:(111/133): python3-kubernetes-11.0.0-2.el8.noar 15 MB/s | 1.2 MB 00:00 2020-11-16T22:21:22.362 INFO:teuthology.orchestra.run.smithi194.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.4 MB/s | 49 kB 00:00 2020-11-16T22:21:22.383 INFO:teuthology.orchestra.run.smithi194.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-16T22:21:22.404 INFO:teuthology.orchestra.run.smithi194.stdout:(114/133): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-11-16T22:21:22.424 INFO:teuthology.orchestra.run.smithi194.stdout:(115/133): python3-portend-2.6-1.el8.noarch.rpm 793 kB/s | 16 kB 00:00 2020-11-16T22:21:22.427 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2020-11-16T22:21:22.428 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2020-11-16T22:21:22.444 INFO:teuthology.orchestra.run.smithi194.stdout:(116/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.4 MB/s | 48 kB 00:00 2020-11-16T22:21:22.467 INFO:teuthology.orchestra.run.smithi194.stdout:(117/133): python3-repoze-lru-0.7-6.el8.noarch. 1.7 MB/s | 34 kB 00:00 2020-11-16T22:21:22.485 INFO:teuthology.orchestra.run.smithi194.stdout:(118/133): python3-routes-2.4.1-12.el8.noarch.r 9.2 MB/s | 196 kB 00:00 2020-11-16T22:21:22.506 INFO:teuthology.orchestra.run.smithi194.stdout:(119/133): python3-rsa-3.4.2-15.el8.noarch.rpm 3.5 MB/s | 73 kB 00:00 2020-11-16T22:21:22.525 INFO:teuthology.orchestra.run.smithi194.stdout:(120/133): python3-simplegeneric-0.8.1-17.el8.n 953 kB/s | 19 kB 00:00 2020-11-16T22:21:22.545 INFO:teuthology.orchestra.run.smithi194.stdout:(121/133): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-11-16T22:21:22.565 INFO:teuthology.orchestra.run.smithi194.stdout:(122/133): python3-tempora-1.14.1-5.el8.noarch. 1.4 MB/s | 29 kB 00:00 2020-11-16T22:21:22.585 INFO:teuthology.orchestra.run.smithi194.stdout:(123/133): python3-trustme-0.6.0-4.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2020-11-16T22:21:22.606 INFO:teuthology.orchestra.run.smithi194.stdout:(124/133): python3-waitress-1.2.1-2.el8.1.noarc 11 MB/s | 241 kB 00:00 2020-11-16T22:21:22.628 INFO:teuthology.orchestra.run.smithi194.stdout:(125/133): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-11-16T22:21:22.648 INFO:teuthology.orchestra.run.smithi194.stdout:(126/133): python3-websocket-client-0.56.0-5.el 3.0 MB/s | 61 kB 00:00 2020-11-16T22:21:22.668 INFO:teuthology.orchestra.run.smithi194.stdout:(127/133): python3-webtest-2.0.33-1.el8.noarch. 4.2 MB/s | 85 kB 00:00 2020-11-16T22:21:22.688 INFO:teuthology.orchestra.run.smithi194.stdout:(128/133): python3-zc-lockfile-2.0-2.el8.noarch 1.1 MB/s | 23 kB 00:00 2020-11-16T22:21:22.708 INFO:teuthology.orchestra.run.smithi194.stdout:(129/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.4 MB/s | 69 kB 00:00 2020-11-16T22:21:22.822 INFO:teuthology.orchestra.run.smithi194.stdout:(130/133): ceph-mgr-diskprediction-local-16.0.0 4.5 MB/s | 8.6 MB 00:01 2020-11-16T22:21:23.274 INFO:teuthology.orchestra.run.smithi194.stdout:(131/133): librbd1-16.0.0-7198.gb637909a.el8.x8 5.3 MB/s | 2.4 MB 00:00 2020-11-16T22:21:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2020-11-16T22:21:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:23.478 INFO:teuthology.orchestra.run.smithi194.stdout:(132/133): librados2-16.0.0-7198.gb637909a.el8. 4.8 MB/s | 3.7 MB 00:00 2020-11-16T22:21:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-16T22:21:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-16T22:21:23.910 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2020-11-16T22:21:23.910 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2020-11-16T22:21:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-16T22:21:24.071 INFO:teuthology.orchestra.run.smithi194.stdout:(133/133): ceph-test-16.0.0-7198.gb637909a.el8. 5.9 MB/s | 47 MB 00:07 2020-11-16T22:21:24.075 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2020-11-16T22:21:24.075 INFO:teuthology.orchestra.run.smithi194.stdout:Total 10 MB/s | 196 MB 00:19 2020-11-16T22:21:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:24.137 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2020-11-16T22:21:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:24.299 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2020-11-16T22:21:24.299 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2020-11-16T22:21:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-16T22:21:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:24.686 INFO:teuthology.orchestra.run.smithi173.stdout:(133/133): ceph-test-16.0.0-7198.gb637909a.el8. 6.1 MB/s | 47 MB 00:07 2020-11-16T22:21:24.689 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2020-11-16T22:21:24.690 INFO:teuthology.orchestra.run.smithi173.stdout:Total 9.8 MB/s | 196 MB 00:20 2020-11-16T22:21:24.752 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2020-11-16T22:21:24.874 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:24.914 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2020-11-16T22:21:24.915 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2020-11-16T22:21:25.036 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:25.288 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:25.528 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:25.625 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:25.770 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:25.780 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2020-11-16T22:21:25.780 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2020-11-16T22:21:25.847 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:26.298 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:21:26.377 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2020-11-16T22:21:26.378 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2020-11-16T22:21:26.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:26.579 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:26.816 INFO:teuthology.orchestra.run.smithi138.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-16T22:21:26.989 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-16T22:21:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-16T22:21:27.368 INFO:teuthology.orchestra.run.smithi138.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-16T22:21:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:27.649 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:27.851 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-16T22:21:27.979 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-16T22:21:28.143 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-16T22:21:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-16T22:21:28.478 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:28.608 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:28.744 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:29.265 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2020-11-16T22:21:29.280 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:29.403 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-16T22:21:29.874 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-16T22:21:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:30.052 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:30.106 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-16T22:21:30.274 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:30.331 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:30.560 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-16T22:21:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2020-11-16T22:21:30.985 INFO:teuthology.orchestra.run.smithi194.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:31.040 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:31.052 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-16T22:21:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:31.227 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-16T22:21:31.258 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-16T22:21:31.357 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:31.379 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-16T22:21:31.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-16T22:21:31.506 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2020-11-16T22:21:31.570 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-16T22:21:31.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-16T22:21:31.619 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:31.689 INFO:teuthology.orchestra.run.smithi173.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:31.712 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-16T22:21:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:21:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:31.894 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-16T22:21:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 31/140 2020-11-16T22:21:31.997 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:31.999 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-16T22:21:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-16T22:21:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 32/140 2020-11-16T22:21:32.162 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:32.182 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:32.256 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-16T22:21:32.275 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:32.398 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:32.499 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-16T22:21:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:32.669 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-16T22:21:32.712 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:32.760 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:21:32.900 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:32.957 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:32.959 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:33.020 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-16T22:21:33.106 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:33.270 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-16T22:21:33.299 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-16T22:21:33.311 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-16T22:21:33.431 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:33.479 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:33.555 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-16T22:21:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:33.576 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-16T22:21:33.674 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 8/140 2020-11-16T22:21:33.727 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-16T22:21:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 9/140 2020-11-16T22:21:34.030 INFO:teuthology.orchestra.run.smithi122.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-16T22:21:34.041 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:21:34.198 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:34.279 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:34.319 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:34.563 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-16T22:21:34.573 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 10/140 2020-11-16T22:21:34.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-16T22:21:34.875 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-16T22:21:35.169 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:21:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-16T22:21:35.203 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-16T22:21:35.323 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-16T22:21:35.390 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:35.499 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:21:35.535 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:35.555 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-16T22:21:35.685 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:35.727 INFO:teuthology.orchestra.run.smithi173.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-16T22:21:35.760 INFO:teuthology.orchestra.run.smithi194.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-16T22:21:35.909 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-16T22:21:35.911 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-16T22:21:36.017 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-16T22:21:36.246 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-16T22:21:36.246 INFO:teuthology.orchestra.run.smithi173.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-16T22:21:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-16T22:21:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-16T22:21:36.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:36.462 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-16T22:21:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 17/140 2020-11-16T22:21:36.552 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-16T22:21:36.670 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-16T22:21:36.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-16T22:21:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-16T22:21:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-16T22:21:36.872 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-16T22:21:36.912 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-16T22:21:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 42/140 2020-11-16T22:21:37.131 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 43/140 2020-11-16T22:21:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-16T22:21:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:37.444 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 44/140 2020-11-16T22:21:37.519 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-16T22:21:37.592 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-16T22:21:37.706 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:37.722 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:37.762 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-16T22:21:37.811 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:37.988 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-16T22:21:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-16T22:21:38.192 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-16T22:21:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-16T22:21:38.348 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:38.353 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-16T22:21:38.358 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-16T22:21:38.420 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:38.530 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-16T22:21:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-16T22:21:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-16T22:21:38.745 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-16T22:21:38.850 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-16T22:21:38.871 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-16T22:21:38.944 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:21:38.980 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:39.036 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-16T22:21:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 31/140 2020-11-16T22:21:39.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-16T22:21:39.265 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:39.326 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 32/140 2020-11-16T22:21:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-16T22:21:39.537 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-16T22:21:39.578 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-16T22:21:39.650 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-16T22:21:39.720 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-16T22:21:39.783 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-16T22:21:39.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-16T22:21:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-16T22:21:39.971 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-16T22:21:40.092 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-16T22:21:40.116 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-16T22:21:40.195 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-16T22:21:40.274 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:21:40.278 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-16T22:21:40.295 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-16T22:21:40.334 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-16T22:21:40.416 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-16T22:21:40.432 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 31/140 2020-11-16T22:21:40.528 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-16T22:21:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-16T22:21:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-16T22:21:40.646 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 32/140 2020-11-16T22:21:40.694 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:21:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-16T22:21:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 31/140 2020-11-16T22:21:40.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-16T22:21:40.959 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-16T22:21:40.999 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 32/140 2020-11-16T22:21:41.189 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:41.474 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-16T22:21:41.572 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-16T22:21:41.737 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-16T22:21:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-16T22:21:41.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-16T22:21:41.937 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-16T22:21:42.109 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-16T22:21:42.172 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-16T22:21:42.252 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-16T22:21:42.295 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-16T22:21:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-16T22:21:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-16T22:21:42.666 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-16T22:21:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:42.906 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-16T22:21:43.088 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-16T22:21:43.134 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-16T22:21:43.482 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-16T22:21:43.776 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-16T22:21:44.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-16T22:21:44.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-16T22:21:44.320 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-16T22:21:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:44.501 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-16T22:21:44.554 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-16T22:21:44.663 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-16T22:21:44.700 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-16T22:21:44.713 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 42/140 2020-11-16T22:21:44.728 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-16T22:21:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-16T22:21:44.918 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 43/140 2020-11-16T22:21:45.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-16T22:21:45.045 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-16T22:21:45.088 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-16T22:21:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 44/140 2020-11-16T22:21:45.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-16T22:21:45.275 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-16T22:21:45.317 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-16T22:21:45.324 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-16T22:21:45.341 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-16T22:21:45.463 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-16T22:21:45.499 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-16T22:21:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-16T22:21:45.538 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-16T22:21:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-16T22:21:45.759 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-16T22:21:45.759 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 42/140 2020-11-16T22:21:45.789 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-16T22:21:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-16T22:21:45.987 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:21:45.989 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 43/140 2020-11-16T22:21:46.013 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-16T22:21:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 42/140 2020-11-16T22:21:46.146 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-16T22:21:46.196 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-16T22:21:46.205 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 44/140 2020-11-16T22:21:46.317 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 43/140 2020-11-16T22:21:46.347 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-16T22:21:46.384 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-16T22:21:46.393 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-16T22:21:46.514 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 44/140 2020-11-16T22:21:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-16T22:21:46.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-16T22:21:46.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-16T22:21:46.544 INFO:teuthology.orchestra.run.smithi138.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-16T22:21:46.544 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:21:46.562 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-16T22:21:46.704 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-16T22:21:46.756 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-16T22:21:46.830 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-16T22:21:46.861 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-16T22:21:46.882 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-16T22:21:46.983 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-16T22:21:47.034 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-16T22:21:47.108 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-16T22:21:47.109 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-16T22:21:47.133 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-16T22:21:47.183 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-16T22:21:47.286 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-16T22:21:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-16T22:21:47.396 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-16T22:21:47.485 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-16T22:21:47.553 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-16T22:21:47.686 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-16T22:21:47.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-16T22:21:47.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-16T22:21:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-16T22:21:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-16T22:21:47.924 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-16T22:21:47.929 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-16T22:21:47.935 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-16T22:21:48.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-16T22:21:48.122 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-16T22:21:48.181 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-16T22:21:48.248 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-16T22:21:48.273 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-16T22:21:48.324 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-16T22:21:48.408 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-16T22:21:48.461 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-16T22:21:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-16T22:21:48.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-16T22:21:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-16T22:21:48.642 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-16T22:21:48.652 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-16T22:21:48.661 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-16T22:21:48.719 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-16T22:21:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-16T22:21:48.868 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-16T22:21:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-16T22:21:49.056 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-16T22:21:49.098 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-16T22:21:49.141 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-16T22:21:49.252 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-16T22:21:49.295 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-16T22:21:49.319 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-16T22:21:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-16T22:21:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-16T22:21:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-16T22:21:49.640 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-16T22:21:49.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-16T22:21:49.748 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-16T22:21:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-16T22:21:49.886 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-16T22:21:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-16T22:21:50.019 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-16T22:21:50.200 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-16T22:21:50.257 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-16T22:21:50.395 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-16T22:21:50.412 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-16T22:21:50.503 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-16T22:21:50.612 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-16T22:21:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-16T22:21:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-16T22:21:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-16T22:21:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-16T22:21:50.877 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-16T22:21:50.986 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-16T22:21:51.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-16T22:21:51.095 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-16T22:21:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-16T22:21:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-16T22:21:51.299 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-16T22:21:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-16T22:21:51.507 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-16T22:21:51.703 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-16T22:21:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-16T22:21:51.971 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-16T22:21:52.086 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-16T22:21:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-16T22:21:52.306 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-16T22:21:52.345 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:21:52.574 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-16T22:21:52.579 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-16T22:21:52.599 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:21:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-16T22:21:52.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-16T22:21:52.964 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-16T22:21:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-16T22:21:53.209 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-16T22:21:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:21:53.356 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-16T22:21:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-16T22:21:53.502 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-16T22:21:53.587 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:21:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-16T22:21:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:21:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-16T22:21:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:21:53.963 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-16T22:21:54.009 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-16T22:21:54.038 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-16T22:21:54.144 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-16T22:21:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:21:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-16T22:21:54.273 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-16T22:21:54.404 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-16T22:21:54.449 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-16T22:21:54.473 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-16T22:21:54.660 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-16T22:21:54.696 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-16T22:21:54.740 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-16T22:21:54.890 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-16T22:21:54.890 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-16T22:21:54.890 INFO:teuthology.orchestra.run.smithi122.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-16T22:21:54.891 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:21:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:21:54.913 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-16T22:21:55.046 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-16T22:21:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-16T22:21:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-16T22:21:55.327 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-16T22:21:55.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-16T22:21:55.541 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-16T22:21:55.552 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-16T22:21:55.552 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-16T22:21:55.553 INFO:teuthology.orchestra.run.smithi194.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-16T22:21:55.553 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:21:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-16T22:21:55.766 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:21:55.815 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-16T22:21:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-16T22:21:55.932 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-16T22:21:56.135 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-16T22:21:56.142 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-16T22:21:56.204 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-16T22:21:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-16T22:21:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-16T22:21:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-16T22:21:56.389 INFO:teuthology.orchestra.run.smithi173.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-16T22:21:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:21:56.520 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-16T22:21:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-16T22:21:56.740 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-16T22:21:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-16T22:21:56.955 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-16T22:21:56.982 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-16T22:21:57.187 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-16T22:21:57.207 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-16T22:21:57.257 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-16T22:21:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-16T22:21:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-16T22:21:57.623 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-16T22:21:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-16T22:21:57.892 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-16T22:21:58.210 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-16T22:21:58.478 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-16T22:21:58.620 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-16T22:22:01.139 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:01.420 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:02.107 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:02.298 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:02.310 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:02.552 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:02.694 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:02.703 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:03.230 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 96/140 2020-11-16T22:22:03.560 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:03.640 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 97/140 2020-11-16T22:22:03.777 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:10.819 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2020-11-16T22:22:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2020-11-16T22:22:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2020-11-16T22:22:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /run 2020-11-16T22:22:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2020-11-16T22:22:10.821 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2020-11-16T22:22:10.821 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2020-11-16T22:22:10.821 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2020-11-16T22:22:10.821 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /dev 2020-11-16T22:22:10.821 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:11.171 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:11.597 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:11.768 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:13.209 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:13.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:13.441 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-16T22:22:13.832 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-16T22:22:13.979 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:14.329 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-16T22:22:15.439 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-16T22:22:15.554 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 107/140 2020-11-16T22:22:15.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 108/140 2020-11-16T22:22:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:17.317 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:17.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:18.357 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:18.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:19.422 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /dev 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /run 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2020-11-16T22:22:19.423 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2020-11-16T22:22:19.424 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /dev 2020-11-16T22:22:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:19.636 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:19.767 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:19.768 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2020-11-16T22:22:19.768 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2020-11-16T22:22:19.768 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2020-11-16T22:22:19.768 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /run 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2020-11-16T22:22:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:19.803 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:19.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:20.121 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 113/140 2020-11-16T22:22:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:20.505 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:20.534 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:20.553 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 98/140 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /dev 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /run 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2020-11-16T22:22:20.688 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2020-11-16T22:22:20.689 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2020-11-16T22:22:20.689 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2020-11-16T22:22:20.689 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /dev 2020-11-16T22:22:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:20.744 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:21.058 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:21.186 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:21.203 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:21.369 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 116/140 2020-11-16T22:22:21.492 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 99/140 2020-11-16T22:22:21.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:21.673 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:22.198 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:22.307 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-16T22:22:22.528 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-16T22:22:22.677 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:22.709 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:22.711 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:22.789 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:22.936 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-16T22:22:23.078 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-16T22:22:23.115 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 100/140 2020-11-16T22:22:23.157 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-16T22:22:23.334 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-16T22:22:23.337 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:23.693 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-16T22:22:23.744 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 107/140 2020-11-16T22:22:23.777 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 101/140 2020-11-16T22:22:23.905 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-16T22:22:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 107/140 2020-11-16T22:22:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 108/140 2020-11-16T22:22:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 108/140 2020-11-16T22:22:24.474 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-16T22:22:24.711 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-16T22:22:24.882 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:24.968 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-16T22:22:25.214 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-16T22:22:25.483 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-16T22:22:25.555 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 107/140 2020-11-16T22:22:25.658 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:25.824 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 108/140 2020-11-16T22:22:25.844 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:25.871 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:26.045 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:26.059 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:26.275 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:26.764 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 118/140 2020-11-16T22:22:26.891 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:27.118 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:27.151 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:27.217 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 109/140 2020-11-16T22:22:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:27.573 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:27.864 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:28.035 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:28.108 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:28.186 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:28.417 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 110/140 2020-11-16T22:22:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:28.591 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:28.633 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:28.713 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:28.714 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:28.936 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 113/140 2020-11-16T22:22:28.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 121/140 2020-11-16T22:22:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 113/140 2020-11-16T22:22:29.159 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:29.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-16T22:22:29.187 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:29.300 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:29.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-16T22:22:29.328 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:29.454 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 124/140 2020-11-16T22:22:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:29.537 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 116/140 2020-11-16T22:22:29.669 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 111/140 2020-11-16T22:22:29.798 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-16T22:22:30.168 INFO:teuthology.orchestra.run.smithi138.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-16T22:22:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:30.329 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 116/140 2020-11-16T22:22:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 127/140 2020-11-16T22:22:30.382 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 112/140 2020-11-16T22:22:30.558 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 128/140 2020-11-16T22:22:30.751 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 113/140 2020-11-16T22:22:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:30.909 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 114/140 2020-11-16T22:22:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 115/140 2020-11-16T22:22:31.189 INFO:teuthology.orchestra.run.smithi138.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-16T22:22:31.358 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 116/140 2020-11-16T22:22:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-16T22:22:31.531 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:31.584 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:31.686 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:32.111 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-16T22:22:32.255 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-16T22:22:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-16T22:22:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-16T22:22:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:32.854 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:32.854 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:33.138 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:33.139 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:34.012 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:34.013 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-16T22:22:34.013 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-16T22:22:34.013 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-16T22:22:34.013 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-16T22:22:34.014 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 13/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 14/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 15/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 16/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 17/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 18/140 2020-11-16T22:22:34.015 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 19/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 20/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 21/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 22/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 23/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 24/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 25/140 2020-11-16T22:22:34.016 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 26/140 2020-11-16T22:22:34.017 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 27/140 2020-11-16T22:22:34.023 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 28/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 29/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 31/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 32/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 33/140 2020-11-16T22:22:34.024 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 34/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 35/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 36/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 37/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 38/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 39/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 40/140 2020-11-16T22:22:34.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 41/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 42/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 43/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 44/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-16T22:22:34.026 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-16T22:22:34.027 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-16T22:22:34.028 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-16T22:22:34.029 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-16T22:22:34.030 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-16T22:22:34.031 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-16T22:22:34.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-16T22:22:34.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-16T22:22:34.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-16T22:22:34.035 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-16T22:22:34.036 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-16T22:22:34.037 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 137/140 2020-11-16T22:22:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:34.039 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 139/140 2020-11-16T22:22:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:34.600 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:35.633 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-16T22:22:35.633 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout:Downgraded: 2020-11-16T22:22:35.634 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.635 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.636 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.637 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.0.4-10.el8.x86_64 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.1-9.el8.x86_64 2020-11-16T22:22:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-16T22:22:35.639 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.640 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.1-6.el8.noarch 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.5-12.el8.x86_64 2020-11-16T22:22:35.641 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-16T22:22:35.642 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-16T22:22:35.643 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-16T22:22:35.644 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-16T22:22:35.645 INFO:teuthology.orchestra.run.smithi138.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-16T22:22:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-16T22:22:35.647 INFO:teuthology.orchestra.run.smithi138.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: leveldb-1.22-1.el8.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-16T22:22:35.648 INFO:teuthology.orchestra.run.smithi138.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-16T22:22:35.649 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-2.6-1.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-16T22:22:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-16T22:22:35.651 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-16T22:22:35.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-16T22:22:35.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-16T22:22:35.652 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-16T22:22:35.652 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-16T22:22:35.652 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2020-11-16T22:22:35.777 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 118/140 2020-11-16T22:22:35.866 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 118/140 2020-11-16T22:22:35.949 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-16T22:22:36.069 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:36.432 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:36.610 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:37.074 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 121/140 2020-11-16T22:22:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-16T22:22:37.886 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 118/140 2020-11-16T22:22:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-16T22:22:38.088 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:38.137 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 124/140 2020-11-16T22:22:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 121/140 2020-11-16T22:22:38.420 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 119/140 2020-11-16T22:22:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-16T22:22:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-16T22:22:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 120/140 2020-11-16T22:22:38.701 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-16T22:22:38.793 INFO:teuthology.orchestra.run.smithi194.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-16T22:22:38.901 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 124/140 2020-11-16T22:22:39.049 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 127/140 2020-11-16T22:22:39.272 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-16T22:22:39.277 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 128/140 2020-11-16T22:22:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:39.539 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 121/140 2020-11-16T22:22:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-16T22:22:39.629 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:39.791 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-16T22:22:39.838 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 127/140 2020-11-16T22:22:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-16T22:22:39.941 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-16T22:22:40.099 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 124/140 2020-11-16T22:22:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 128/140 2020-11-16T22:22:40.113 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-16T22:22:40.357 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-16T22:22:40.571 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:40.678 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:40.786 INFO:teuthology.orchestra.run.smithi122.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-16T22:22:40.846 INFO:teuthology.orchestra.run.smithi173.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-16T22:22:40.994 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-16T22:22:41.009 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:41.028 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:41.052 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 127/140 2020-11-16T22:22:41.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:41.236 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-16T22:22:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 128/140 2020-11-16T22:22:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:41.396 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-16T22:22:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:41.563 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-16T22:22:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:41.598 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7198.gb6379 129/140 2020-11-16T22:22:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-16T22:22:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:41.808 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-16T22:22:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-16T22:22:41.944 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-16T22:22:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:42.135 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-16T22:22:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-16T22:22:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:42.337 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-16T22:22:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:42.420 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-16T22:22:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:42.646 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:42.665 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-16T22:22:42.822 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-16T22:22:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:42.958 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-16T22:22:43.137 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:43.137 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-16T22:22:43.138 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-16T22:22:43.138 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-16T22:22:43.138 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-16T22:22:43.138 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-16T22:22:43.138 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:22:43.139 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 13/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 14/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 15/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 16/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 17/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 18/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 19/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 20/140 2020-11-16T22:22:43.140 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 21/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 22/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 23/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 24/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 25/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 26/140 2020-11-16T22:22:43.141 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 27/140 2020-11-16T22:22:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 28/140 2020-11-16T22:22:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 29/140 2020-11-16T22:22:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 31/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 32/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 33/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 34/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 35/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 36/140 2020-11-16T22:22:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 37/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 38/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 39/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 40/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 41/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 42/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 43/140 2020-11-16T22:22:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 44/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-16T22:22:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-16T22:22:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-16T22:22:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-16T22:22:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-16T22:22:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-16T22:22:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-16T22:22:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-16T22:22:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-16T22:22:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-16T22:22:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-16T22:22:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-16T22:22:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 137/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 139/140 2020-11-16T22:22:43.163 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-16T22:22:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-16T22:22:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:43.565 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:43.566 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-16T22:22:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:43.847 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-16T22:22:43.848 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:22:43.849 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 13/140 2020-11-16T22:22:43.850 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 14/140 2020-11-16T22:22:43.850 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 15/140 2020-11-16T22:22:43.850 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 16/140 2020-11-16T22:22:43.851 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 17/140 2020-11-16T22:22:43.851 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 18/140 2020-11-16T22:22:43.851 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 19/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 20/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 21/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 22/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 23/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 24/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 25/140 2020-11-16T22:22:43.852 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 26/140 2020-11-16T22:22:43.853 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 27/140 2020-11-16T22:22:43.853 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 28/140 2020-11-16T22:22:43.853 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 29/140 2020-11-16T22:22:43.853 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:22:43.853 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 31/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 32/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 33/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 34/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 35/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 36/140 2020-11-16T22:22:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 37/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 38/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 39/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 40/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 41/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 42/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 43/140 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 44/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-16T22:22:43.863 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-16T22:22:43.864 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-16T22:22:43.865 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-16T22:22:43.866 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-16T22:22:43.867 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-16T22:22:43.868 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-16T22:22:43.869 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-16T22:22:43.870 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-16T22:22:43.871 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-16T22:22:43.872 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-16T22:22:43.873 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 137/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:43.874 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 139/140 2020-11-16T22:22:44.641 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-16T22:22:44.641 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-16T22:22:44.641 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-16T22:22:44.642 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:16.0.0-7198.gb637909a.el8.x86_64 11/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 12/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 13/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 14/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7198.gb6379 15/140 2020-11-16T22:22:44.643 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 16/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 17/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 18/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 19/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 20/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 21/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 22/140 2020-11-16T22:22:44.644 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 23/140 2020-11-16T22:22:44.645 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_ 24/140 2020-11-16T22:22:44.645 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 25/140 2020-11-16T22:22:44.645 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:16.0.0-7198.gb637909a.el8.x86_6 26/140 2020-11-16T22:22:44.645 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86 27/140 2020-11-16T22:22:44.651 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 28/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7198.gb637909a.el 29/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:16.0.0-7198.gb637909a.el8. 30/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_6 31/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 32/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 33/140 2020-11-16T22:22:44.652 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 34/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 35/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 36/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 37/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7198.gb637909a. 38/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noa 39/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.n 40/140 2020-11-16T22:22:44.653 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7198.gb63 41/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el 42/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 43/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.e 44/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-16T22:22:44.654 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-16T22:22:44.655 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-16T22:22:44.656 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-16T22:22:44.657 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-16T22:22:44.658 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-16T22:22:44.659 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-16T22:22:44.660 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-16T22:22:44.661 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-16T22:22:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-16T22:22:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-16T22:22:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-16T22:22:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.0.0-7198.gb637909a.el8.x86_64 137/140 2020-11-16T22:22:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-16T22:22:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 139/140 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.753 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout:Downgraded: 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:44.754 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.755 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.756 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.0.4-10.el8.x86_64 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.1-9.el8.x86_64 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-16T22:22:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.1-6.el8.noarch 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.5-12.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-16T22:22:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-16T22:22:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-16T22:22:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-16T22:22:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-16T22:22:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-16T22:22:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: leveldb-1.22-1.el8.x86_64 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-16T22:22:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-16T22:22:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-16T22:22:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-2.6-1.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-16T22:22:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-16T22:22:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-16T22:22:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:44.773 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2020-11-16T22:22:44.887 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:45.594 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-16T22:22:45.595 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:45.595 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2020-11-16T22:22:45.595 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.595 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.595 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout:Downgraded: 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2020-11-16T22:22:45.596 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.597 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.598 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.599 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.0.4-10.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.1-9.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.601 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:45.602 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.1-6.el8.noarch 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: jq-1.5-12.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-16T22:22:45.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:45.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-16T22:22:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-16T22:22:45.606 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-16T22:22:45.607 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-16T22:22:45.608 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: leveldb-1.22-1.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-16T22:22:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-16T22:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-16T22:22:45.611 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-2.6-1.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-16T22:22:45.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-16T22:22:45.613 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:45.614 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2020-11-16T22:22:45.792 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:46.342 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-16T22:22:46.342 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout:Downgraded: 2020-11-16T22:22:46.343 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.344 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.346 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.0.4-10.el8.x86_64 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.1-9.el8.x86_64 2020-11-16T22:22:46.347 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-16T22:22:46.348 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:16.0.0-7198.gb637909a.el8.x86_64 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:16.0.0-7198.gb637909a.el8.noarch 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.1-6.el8.noarch 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: jq-1.5-12.el8.x86_64 2020-11-16T22:22:46.350 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-16T22:22:46.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-16T22:22:46.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-16T22:22:46.353 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-16T22:22:46.354 INFO:teuthology.orchestra.run.smithi173.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-16T22:22:46.355 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-16T22:22:46.356 INFO:teuthology.orchestra.run.smithi173.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: leveldb-1.22-1.el8.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-16T22:22:46.357 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-16T22:22:46.358 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-2.6-1.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-16T22:22:46.359 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-16T22:22:46.360 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:46.361 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2020-11-16T22:22:46.502 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:46.502 INFO:teuthology.orchestra.run.smithi138:> sudo lsb_release -is 2020-11-16T22:22:46.559 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS 2020-11-16T22:22:46.560 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:22:46.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:22:46.729 INFO:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-16T22:22:46.750 INFO:teuthology.orchestra.run.smithi138.stdout:16.0.0-7198.gb637909a.el8 2020-11-16T22:22:46.751 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7198.gb637909a.el8 2020-11-16T22:22:46.751 INFO:teuthology.task.install:The correct ceph version 16.0.0-7198.gb637909a is installed. 2020-11-16T22:22:46.752 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-16T22:22:46.803 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS 2020-11-16T22:22:46.804 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:22:46.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:22:46.964 INFO:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-16T22:22:46.982 INFO:teuthology.orchestra.run.smithi173.stdout:16.0.0-7198.gb637909a.el8 2020-11-16T22:22:46.983 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7198.gb637909a.el8 2020-11-16T22:22:46.983 INFO:teuthology.task.install:The correct ceph version 16.0.0-7198.gb637909a is installed. 2020-11-16T22:22:46.983 INFO:teuthology.orchestra.run.smithi194:> sudo lsb_release -is 2020-11-16T22:22:47.038 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS 2020-11-16T22:22:47.039 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:22:47.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:22:47.204 INFO:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-16T22:22:47.225 INFO:teuthology.orchestra.run.smithi194.stdout:16.0.0-7198.gb637909a.el8 2020-11-16T22:22:47.225 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7198.gb637909a.el8 2020-11-16T22:22:47.225 INFO:teuthology.task.install:The correct ceph version 16.0.0-7198.gb637909a is installed. 2020-11-16T22:22:47.226 INFO:teuthology.orchestra.run.smithi122:> sudo lsb_release -is 2020-11-16T22:22:47.279 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS 2020-11-16T22:22:47.280 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-16T22:22:47.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b637909a9153a419511f943e4eee7800ad3bbb99 2020-11-16T22:22:47.439 INFO:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-16T22:22:47.460 INFO:teuthology.orchestra.run.smithi122.stdout:16.0.0-7198.gb637909a.el8 2020-11-16T22:22:47.460 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7198.gb637909a.el8 2020-11-16T22:22:47.460 INFO:teuthology.task.install:The correct ceph version 16.0.0-7198.gb637909a is installed. 2020-11-16T22:22:47.461 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-16T22:22:47.461 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:47.461 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.477 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.491 INFO:teuthology.orchestra.run.smithi138.stderr:24+1 records in 2020-11-16T22:22:47.492 INFO:teuthology.orchestra.run.smithi138.stderr:24+1 records out 2020-11-16T22:22:47.492 INFO:teuthology.orchestra.run.smithi138.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.5965e-05 s, 167 MB/s 2020-11-16T22:22:47.493 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:47.493 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.507 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.520 INFO:teuthology.orchestra.run.smithi173.stderr:24+1 records in 2020-11-16T22:22:47.521 INFO:teuthology.orchestra.run.smithi173.stderr:24+1 records out 2020-11-16T22:22:47.521 INFO:teuthology.orchestra.run.smithi173.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.2971e-05 s, 174 MB/s 2020-11-16T22:22:47.521 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:47.522 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.537 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.552 INFO:teuthology.orchestra.run.smithi194.stderr:24+1 records in 2020-11-16T22:22:47.553 INFO:teuthology.orchestra.run.smithi194.stderr:24+1 records out 2020-11-16T22:22:47.553 INFO:teuthology.orchestra.run.smithi194.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.5341e-05 s, 168 MB/s 2020-11-16T22:22:47.554 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:47.554 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.568 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:22:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:24+1 records in 2020-11-16T22:22:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:24+1 records out 2020-11-16T22:22:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.8033e-05 s, 162 MB/s 2020-11-16T22:22:47.584 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-16T22:22:47.584 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:47.585 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.600 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.615 INFO:teuthology.orchestra.run.smithi138.stderr:5+1 records in 2020-11-16T22:22:47.615 INFO:teuthology.orchestra.run.smithi138.stderr:5+1 records out 2020-11-16T22:22:47.615 INFO:teuthology.orchestra.run.smithi138.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.6098e-05 s, 66.3 MB/s 2020-11-16T22:22:47.616 INFO:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:22:47.686 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:47.686 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.699 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.712 INFO:teuthology.orchestra.run.smithi173.stderr:5+1 records in 2020-11-16T22:22:47.712 INFO:teuthology.orchestra.run.smithi173.stderr:5+1 records out 2020-11-16T22:22:47.713 INFO:teuthology.orchestra.run.smithi173.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 3.6866e-05 s, 82.9 MB/s 2020-11-16T22:22:47.713 INFO:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:22:47.780 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:47.780 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.795 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.810 INFO:teuthology.orchestra.run.smithi194.stderr:5+1 records in 2020-11-16T22:22:47.810 INFO:teuthology.orchestra.run.smithi194.stderr:5+1 records out 2020-11-16T22:22:47.811 INFO:teuthology.orchestra.run.smithi194.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.5353e-05 s, 67.4 MB/s 2020-11-16T22:22:47.811 INFO:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:22:47.881 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:47.881 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.897 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:22:47.911 INFO:teuthology.orchestra.run.smithi122.stderr:5+1 records in 2020-11-16T22:22:47.911 INFO:teuthology.orchestra.run.smithi122.stderr:5+1 records out 2020-11-16T22:22:47.911 INFO:teuthology.orchestra.run.smithi122.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 4.2507e-05 s, 71.9 MB/s 2020-11-16T22:22:47.912 INFO:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:22:47.981 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-16T22:22:47.981 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:47.982 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:47.997 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.012 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:22:48.012 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:22:48.012 INFO:teuthology.orchestra.run.smithi138.stderr:299 bytes copied, 5.4692e-05 s, 5.5 MB/s 2020-11-16T22:22:48.013 INFO:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:22:48.083 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:48.083 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.096 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.110 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:22:48.110 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:22:48.110 INFO:teuthology.orchestra.run.smithi173.stderr:299 bytes copied, 2.734e-05 s, 10.9 MB/s 2020-11-16T22:22:48.111 INFO:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:22:48.178 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:48.178 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.194 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.208 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:22:48.208 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:22:48.208 INFO:teuthology.orchestra.run.smithi194.stderr:299 bytes copied, 3.1491e-05 s, 9.5 MB/s 2020-11-16T22:22:48.209 INFO:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:22:48.279 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:48.279 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:22:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:22:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:22:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:299 bytes copied, 2.8114e-05 s, 10.6 MB/s 2020-11-16T22:22:48.309 INFO:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:22:48.379 INFO:teuthology.run_tasks:Running task ceph... 2020-11-16T22:22:48.436 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-16T22:22:48.437 INFO:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2020-11-16T22:22:48.439 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2020-11-16T22:22:48.441 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2020-11-16T22:22:48.444 INFO:teuthology.orchestra.run.smithi194:> sudo chmod 777 /var/log/ceph 2020-11-16T22:22:48.474 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-16T22:22:48.474 INFO:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:22:48.508 INFO:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:22:48.512 INFO:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:22:48.514 INFO:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:22:48.542 INFO:tasks.ceph:Creating extra log directories... 2020-11-16T22:22:48.543 INFO:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:22:48.576 INFO:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:22:48.582 INFO:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:22:48.584 INFO:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:22:48.621 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-16T22:22:48.622 INFO:tasks.ceph:config {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-16T22:22:48.622 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024', 'branch': 'master', 'description': 'powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5631024', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi', 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'master'}, 'ceph': {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'master', 'sha1': '6441771be52a0f47b9e118f8bca335f29dfda438'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 80, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'], ['osd.0'], ['osd.1'], ['osd.2']], 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99', 'sleep_before_teardown': 0, 'suite': 'powercycle', 'suite_branch': 'master', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_master/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': '6441771be52a0f47b9e118f8bca335f29dfda438', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (4, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}}, {'ceph': None}, {'thrashosds': {'chance_down': 1.0, 'powercycle': True, 'timeout': 600}}, {'ceph-fuse': None}, {'workunit': {'clients': {'all': ['suites/fsx.sh']}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_11:15:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.11568', 'targets': {'smithi138.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrzrkjUHYhtPv1T+itDNnKqc54x5/iplG6NjcfGMMJatSRWozQUx5lYwRGndR+OtuOkd2gUTyrpuv56GMAuE/Ubhfo9IXaSBKTy9G2WGFVkF6driH9+s4tldEd6zunrU2Ki6LQMfYzsaLHbXYfqfaLLmrHdSGazHqeQZwQC8Eqq2qYyO6ObG601RyRwIG+qG2oM/bVBxofJ8B0c+lLuBb2500cJB0DTWoB9vIyF2qH1y4+UdQdgIF2WjWdzIv6+oZtPJfA79QP6sY7pPgi42lH5PYU3bNz4BfLZtWnAnk/ACgCVAe+bo5UgrwdMEjBymAcWrx67REJs8IhYbbRFvmExEsJ8Kni0md6sKl73srqRQLv7CsYKtJeswO8X8gePA8ue7wMICNfcvOlkiD0wCMBIKt4K/F3AGIZRYGgJU/vrs5Z0qthPVBnJqasyPNh6vuEKeYo2Sr+P5fNhNzMOt4eJFQfFWdVaJ5Xoq4rU6z3kE8ZpSyeOFLWfFbxRSFlKC0=', 'smithi173.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzAAIpAkgVglbuXrxrMaE7OqMYdaw9gudwLBy71h0P505j0+FXcncvcPPoeQ974acr5wVqZtJgLg5wNgN1hSIDEcpoJU07zD0Q1BymiN5oYzeLof3b/Z9sbUr49VzvfSl1RZkga0SG4LlCvuW6jLYqDLYgsfCXpiifbv8zKCU9xPzfcFyqaTdCN3hsb+WZZ81nNrx4UJQeBW/E4nm+PCcvsnnk6rfNyWvnhtP8CR04TC6M08yDDju2sD97+mR/8mZrwxKekEF5LJemVKfczZ81UUgo3lIHredN0KxM63mKGqNgurl64Xcc6DjU15kVm9glBwAsN4YbS5gsISm5FA7cHWQD4bk3XCsD9X/9/aNWvnpJcC6BkuYQtXN3M7RbmtikTXs66c5BNDVmfHJ0+Q3DfeJayTw5xKdJRcGgdotybX6KFsD02N3Hc7kGBsYRkwN06QoYCm2Tvi6LyBfgPNHVSKJ9fU4d4MqHxnJPFv4O5KsXpoW6oCqP/BT4JR1uTBM=', 'smithi194.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZrIGBqIb1Mtn+cTIQwjPZlxvsAnU+gLH6N2C10IFfPCZT9m/VhTk0Kj8d+Y4og8JGi8BobQE3utjznZBdDjHV2nTyWUiG48isOmJ+vvEv3muKyeDLYInu6l3g7C044N+IdAU1VTTTMK3PMI2FfIJXjsCZTatfp1nUhZ/yYSdJUxzIWS1c37X94M28xoY8iWUzJrsFKoh3aiJ0rY+EczIx6wsuqx9uCZrq3P0kYpeaMETjvgKICnKR4qYVnj8lrTIpsTOcXxxot3v463miQtl4GRfGtABhw8nrKgA0EWg1tuPp2kqaE8XoCs0DgDeZ/HOk8JNZF2xQJ8Y3G/Itp2Zw/x2HlNFtnkyHr9TxeC1ZL30t/gaCUaVLPCaYZpeDO2/dw2nWzIOrqoIg74dHq8RPLJzaJjfODlzk1uVPPByhRng8RKr/i2GJlU7Nuiubwl3Mgz7ep7wN+knMLHNi1vPNWrqzpLrUs2UaXr1ocHO8RX+0SFX2bxJQMbkPalW0m90=', 'smithi122.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqWOvZqtmsS/gx5H+CiHACrR6ZuO+Vm36ArOBrqTKtow87DYWXruyM/dHsRVy7/bfHyte2QXUhouebGs3wj3rQiG6NCML8dERf/K0MX0HnQ1a1XWDvOgtp2eA+msJafhD0ucB8jwSESxX8qbiN8IK8Hlcu6tpNb01GubNFxMyhOYgUyJxwQPC3jZJ0VCu+OXje9TTE5brjUPZ9gewR5oC57FFcraybBM6NIlQ00usoKDO/nlMZDI+rsX6o5YZ5+tOyaVLYEJYrVthKESHMvOdFfpip153NI2jyeHIbHSxg92cz5L3v5bDZEuNglvyKKwRIco5LesoV8idwJHk0fKe1ymHpmyj25Ze7v2XlhWjM1vTsYgda5227BnOON+oN/eahaTqGoljX76ACl0Ja3wEVNWR8c9IB3rSPIty83SOl6O9IKWVQEn3cGwimoe6DpfXMp6FYmoFFovITuL3h/Stq8aFYtyTo2wwMfLkIY7Mq+Yf+I1I3SXQ+WbTyCejDSLM='}} 2020-11-16T22:22:48.623 INFO:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:22:48.645 INFO:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:22:48.651 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:22:48.653 INFO:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:22:48.668 INFO:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:22:48.700 INFO:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:22:48.706 INFO:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:22:48.708 INFO:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:22:48.737 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:48.737 INFO:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:48.785 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:48.786 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:22:48.786 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:22:48.786 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:22:48.786 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:22:48.786 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:22:48.787 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:22:48.787 INFO:teuthology.orchestra.run.smithi173.stderr:72 bytes copied, 3.2092e-05 s, 2.2 MB/s 2020-11-16T22:22:48.787 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:22:48.787 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:22:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:22:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:48.840 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 60017 Links: 1 2020-11-16T22:22:48.840 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:48.840 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:48.841 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-16 22:22:44.290623682 +0000 2020-11-16T22:22:48.841 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-16 22:20:35.674738860 +0000 2020-11-16T22:22:48.841 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-16 22:20:35.674738860 +0000 2020-11-16T22:22:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-16T22:22:48.853 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-16T22:22:48.853 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-16T22:22:48.853 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000227468 s, 2.3 MB/s 2020-11-16T22:22:48.856 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:22:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:22:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:48.910 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 60116 Links: 1 2020-11-16T22:22:48.910 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:48.911 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:48.911 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-16 22:22:44.290623682 +0000 2020-11-16T22:22:48.911 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-16 22:20:35.998732223 +0000 2020-11-16T22:22:48.911 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-16 22:20:35.998732223 +0000 2020-11-16T22:22:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-16T22:22:48.923 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-16T22:22:48.923 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-16T22:22:48.923 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000212321 s, 2.4 MB/s 2020-11-16T22:22:48.926 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:22:48.979 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:22:48.979 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:48.979 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 62894 Links: 1 2020-11-16T22:22:48.979 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:48.980 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:48.980 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-16 22:22:44.290623682 +0000 2020-11-16T22:22:48.980 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-16 22:20:36.330725421 +0000 2020-11-16T22:22:48.980 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-16 22:20:36.330725421 +0000 2020-11-16T22:22:48.980 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-16T22:22:48.992 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-16T22:22:48.992 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-16T22:22:48.992 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000189138 s, 2.7 MB/s 2020-11-16T22:22:48.994 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:22:49.048 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 57250 Links: 1 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-16 22:22:44.290623682 +0000 2020-11-16T22:22:49.049 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-16 22:20:36.647718927 +0000 2020-11-16T22:22:49.050 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-16 22:20:36.647718927 +0000 2020-11-16T22:22:49.050 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-16T22:22:49.061 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-16T22:22:49.061 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-16T22:22:49.062 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000190864 s, 2.7 MB/s 2020-11-16T22:22:49.064 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:49.064 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:49.064 INFO:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:49.078 INFO:teuthology.orchestra.run.smithi194.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:49.079 INFO:teuthology.orchestra.run.smithi194.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:22:49.079 INFO:teuthology.orchestra.run.smithi194.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:22:49.079 INFO:teuthology.orchestra.run.smithi194.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:22:49.079 INFO:teuthology.orchestra.run.smithi194.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:22:49.080 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:22:49.080 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:22:49.080 INFO:teuthology.orchestra.run.smithi194.stderr:72 bytes copied, 3.4606e-05 s, 2.1 MB/s 2020-11-16T22:22:49.080 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:22:49.080 INFO:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:22:49.134 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 58365 Links: 1 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2020-11-16 22:22:42.808431740 +0000 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2020-11-16 22:20:43.921071198 +0000 2020-11-16T22:22:49.135 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2020-11-16 22:20:43.921071198 +0000 2020-11-16T22:22:49.136 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2020-11-16T22:22:49.148 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2020-11-16T22:22:49.149 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2020-11-16T22:22:49.149 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000238805 s, 2.1 MB/s 2020-11-16T22:22:49.155 INFO:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:22:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:22:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.210 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 57299 Links: 1 2020-11-16T22:22:49.210 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.210 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.211 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2020-11-16 22:22:42.809431718 +0000 2020-11-16T22:22:49.211 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2020-11-16 22:20:44.249064433 +0000 2020-11-16T22:22:49.211 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2020-11-16 22:20:44.249064433 +0000 2020-11-16T22:22:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2020-11-16T22:22:49.223 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2020-11-16T22:22:49.223 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2020-11-16T22:22:49.224 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000276267 s, 1.9 MB/s 2020-11-16T22:22:49.226 INFO:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:22:49.280 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 61881 Links: 1 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2020-11-16 22:22:42.809431718 +0000 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2020-11-16 22:20:44.567057874 +0000 2020-11-16T22:22:49.281 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2020-11-16 22:20:44.567057874 +0000 2020-11-16T22:22:49.282 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2020-11-16T22:22:49.294 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2020-11-16T22:22:49.294 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2020-11-16T22:22:49.294 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.00021562 s, 2.4 MB/s 2020-11-16T22:22:49.296 INFO:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:22:49.350 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:22:49.350 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.350 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 62878 Links: 1 2020-11-16T22:22:49.350 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.350 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.351 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2020-11-16 22:22:42.809431718 +0000 2020-11-16T22:22:49.351 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2020-11-16 22:20:44.891051192 +0000 2020-11-16T22:22:49.351 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2020-11-16 22:20:44.891051192 +0000 2020-11-16T22:22:49.351 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2020-11-16T22:22:49.363 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2020-11-16T22:22:49.363 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2020-11-16T22:22:49.363 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000181595 s, 2.8 MB/s 2020-11-16T22:22:49.365 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:49.366 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:49.366 INFO:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:49.381 INFO:teuthology.orchestra.run.smithi122.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:22:49.382 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:22:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:72 bytes copied, 3.4849e-05 s, 2.1 MB/s 2020-11-16T22:22:49.384 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:22:49.384 INFO:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:22:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:22:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 56306 Links: 1 2020-11-16T22:22:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.440 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.440 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2020-11-16 22:22:43.468905065 +0000 2020-11-16T22:22:49.440 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2020-11-16 22:20:35.643021801 +0000 2020-11-16T22:22:49.440 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2020-11-16 22:20:35.643021801 +0000 2020-11-16T22:22:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2020-11-16T22:22:49.454 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2020-11-16T22:22:49.454 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2020-11-16T22:22:49.454 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000238677 s, 2.1 MB/s 2020-11-16T22:22:49.457 INFO:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:22:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 61313 Links: 1 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2020-11-16 22:22:43.468905065 +0000 2020-11-16T22:22:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2020-11-16 22:20:35.978014254 +0000 2020-11-16T22:22:49.514 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2020-11-16 22:20:35.978014254 +0000 2020-11-16T22:22:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2020-11-16T22:22:49.527 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2020-11-16T22:22:49.527 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2020-11-16T22:22:49.527 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000220741 s, 2.3 MB/s 2020-11-16T22:22:49.529 INFO:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:22:49.584 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:22:49.584 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.584 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 62697 Links: 1 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2020-11-16 22:22:43.468905065 +0000 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2020-11-16 22:20:36.290007226 +0000 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2020-11-16 22:20:36.290007226 +0000 2020-11-16T22:22:49.585 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2020-11-16T22:22:49.598 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2020-11-16T22:22:49.598 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2020-11-16T22:22:49.598 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000219855 s, 2.3 MB/s 2020-11-16T22:22:49.600 INFO:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:22:49.655 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:22:49.655 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 61385 Links: 1 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2020-11-16 22:22:43.468905065 +0000 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2020-11-16 22:20:36.662998824 +0000 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2020-11-16 22:20:36.662998824 +0000 2020-11-16T22:22:49.656 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2020-11-16T22:22:49.668 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2020-11-16T22:22:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2020-11-16T22:22:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000215585 s, 2.4 MB/s 2020-11-16T22:22:49.671 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:49.671 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2020-11-16T22:22:49.671 INFO:tasks.ceph:Generating config... 2020-11-16T22:22:49.673 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-16T22:22:49.673 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-16T22:22:49.673 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-16T22:22:49.673 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-16T22:22:49.674 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-16T22:22:49.674 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-16T22:22:49.674 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2020-11-16T22:22:49.674 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2020-11-16T22:22:49.674 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-16T22:22:49.675 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-16T22:22:49.675 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-16T22:22:49.675 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-16T22:22:49.675 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-16T22:22:49.675 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-16T22:22:49.676 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-16T22:22:49.676 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-16T22:22:49.676 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-16T22:22:49.676 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-16T22:22:49.676 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-16T22:22:49.677 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-16T22:22:49.677 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2020-11-16T22:22:49.677 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-16T22:22:49.677 INFO:tasks.ceph:Setting up mon.a... 2020-11-16T22:22:49.678 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-16T22:22:50.016 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.keyring 2020-11-16T22:22:50.018 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-16T22:22:50.066 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.094 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.138'), ('mon.b', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]'), ('mon.c', '[v2:172.21.15.138:3302,v1:172.21.15.138:6791]')] 2020-11-16T22:22:50.094 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}} 2020-11-16T22:22:50.095 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:50.095 INFO:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:22:50.147 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:22:50.148 INFO:teuthology.orchestra.run.smithi138.stderr:5+1 records in 2020-11-16T22:22:50.148 INFO:teuthology.orchestra.run.smithi138.stderr:5+1 records out 2020-11-16T22:22:50.148 INFO:teuthology.orchestra.run.smithi138.stderr:3019 bytes (3.0 kB, 2.9 KiB) copied, 6.4772e-05 s, 46.6 MB/s 2020-11-16T22:22:50.149 INFO:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.138 --addv b '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --addv c '[v2:172.21.15.138:3302,v1:172.21.15.138:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.230 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: generated fsid 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.231 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 0 2020-11-16T22:22:50.231 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.231 INFO:teuthology.orchestra.run.smithi138.stdout:last_changed 2020-11-16T22:22:50.243438+0000 2020-11-16T22:22:50.232 INFO:teuthology.orchestra.run.smithi138.stdout:created 2020-11-16T22:22:50.243438+0000 2020-11-16T22:22:50.232 INFO:teuthology.orchestra.run.smithi138.stdout:min_mon_release 0 (unknown) 2020-11-16T22:22:50.232 INFO:teuthology.orchestra.run.smithi138.stdout:election_strategy: 1 2020-11-16T22:22:50.232 INFO:teuthology.orchestra.run.smithi138.stdout:0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2020-11-16T22:22:50.232 INFO:teuthology.orchestra.run.smithi138.stdout:1: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.b 2020-11-16T22:22:50.233 INFO:teuthology.orchestra.run.smithi138.stdout:2: [v2:172.21.15.138:3302/0,v1:172.21.15.138:6791/0] mon.c 2020-11-16T22:22:50.233 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-16T22:22:50.235 INFO:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:22:50.288 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2963e9ac-8003-490b-a7b3-aee3b9109ae1... 2020-11-16T22:22:50.289 INFO:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:22:50.292 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:22:50.330 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:22:50.333 INFO:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:22:50.347 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2020-11-16T22:22:50.347 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2020-11-16T22:22:50.347 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:22:50.349 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.350 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:22:50.351 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:22:50.352 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2020-11-16T22:22:50.352 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.352 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:22:50.352 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.352 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2020-11-16T22:22:50.353 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]" 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2020-11-16T22:22:50.354 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2020-11-16T22:22:50.355 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2020-11-16T22:22:50.356 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2020-11-16T22:22:50.357 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression algorithm = zstd 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression mode = aggressive 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: debug filestore = 20 2020-11-16T22:22:50.358 INFO:teuthology.orchestra.run.smithi122.stdout: debug journal = 20 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 20 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 25 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:22:50.359 INFO:teuthology.orchestra.run.smithi122.stdout: osd max markdown count = 1000 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2020-11-16T22:22:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2020-11-16T22:22:50.361 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2020-11-16T22:22:50.362 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:22:50.363 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2020-11-16T22:22:50.364 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2020-11-16T22:22:50.364 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2020-11-16T22:22:50.364 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.381 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2020-11-16T22:22:50.382 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2020-11-16T22:22:50.383 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2020-11-16T22:22:50.384 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2020-11-16T22:22:50.385 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:22:50.385 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:22:50.385 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2020-11-16T22:22:50.385 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:22:50.385 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:22:50.386 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2020-11-16T22:22:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2020-11-16T22:22:50.388 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2020-11-16T22:22:50.388 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]" 2020-11-16T22:22:50.388 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.388 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.388 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.389 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2020-11-16T22:22:50.390 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2020-11-16T22:22:50.390 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.390 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2020-11-16T22:22:50.390 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2020-11-16T22:22:50.390 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2020-11-16T22:22:50.391 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.391 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2020-11-16T22:22:50.391 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2020-11-16T22:22:50.391 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.391 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.392 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression algorithm = zstd 2020-11-16T22:22:50.393 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression mode = aggressive 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug filestore = 20 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug journal = 20 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 20 2020-11-16T22:22:50.394 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 25 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: osd max markdown count = 1000 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2020-11-16T22:22:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.396 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2020-11-16T22:22:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:22:50.398 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2020-11-16T22:22:50.399 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2020-11-16T22:22:50.400 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2020-11-16T22:22:50.401 INFO:teuthology.orchestra.run.smithi194.stdout:[global] 2020-11-16T22:22:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: chdir = "" 2020-11-16T22:22:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:22:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: auth supported = cephx 2020-11-16T22:22:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: filestore xattr use omap = true 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: mon clock drift allowed = 1.000 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: osd crush chooseleaf type = 0 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: auth debug = true 2020-11-16T22:22:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on old message = true 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on bug = true 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: mon pg warn max object skew = 0 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default size = 2 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow primary affinity = true 2020-11-16T22:22:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow pg remap = true 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on no sortbitwise = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on too few osds = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:22:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: mon_allow_pool_size_one = true 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: osd default data pool replay window = 5 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: mon allow pool delete = true 2020-11-16T22:22:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: mon cluster log file level = debug 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: debug asserts on shutdown = true 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: mon health detail to clog = false 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: mon host = "172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]" 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: fsid = 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.407 INFO:teuthology.orchestra.run.smithi194.stdout:[osd] 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: osd journal size = 100 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub load threshold = 5.0 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub max interval = 600 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: osd recover clone overlap = true 2020-11-16T22:22:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: osd recovery max chunk = 1048576 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug shutdown = true 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug op order = true 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug verify stray on activate = true 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: osd open classes on start = true 2020-11-16T22:22:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug pg log writeout = true 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: osd map max advance = 10 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: journal zero on create = true 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: filestore apply finisher threads = 3 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: bdev debug aio = true 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug misdirected ops = true 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore block size = 96636764160 2020-11-16T22:22:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore compression algorithm = zstd 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore compression mode = aggressive 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore fsck on mount = True 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluefs = 20 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluestore = 20 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug filestore = 20 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug journal = 20 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 20 2020-11-16T22:22:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: debug osd = 25 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: debug rocksdb = 10 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd full ratio = 0.9 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: osd max markdown count = 1000 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: osd objectstore = bluestore 2020-11-16T22:22:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr] 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: debug mgr = 20 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2020-11-16T22:22:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: mgr/telemetry/nag = false 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout:[mon] 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: debug paxos = 20 2020-11-16T22:22:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon data avail warn = 5 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon mgr mkfs grace = 240 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd reporter subtree level = osd 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd prime pg temp = true 2020-11-16T22:22:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout:[client] 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: rgw cache enabled = true 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable ops log = true 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable usage log = true 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:22:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.a] 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.b] 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.c] 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.a] 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-16T22:22:50.418 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-16T22:22:50.419 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-16T22:22:50.420 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.421 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-16T22:22:50.422 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.423 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-16T22:22:50.424 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]" 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-16T22:22:50.425 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.426 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-16T22:22:50.427 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.428 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-16T22:22:50.429 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore compression algorithm = zstd 2020-11-16T22:22:50.430 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore compression mode = aggressive 2020-11-16T22:22:50.430 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-16T22:22:50.430 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-16T22:22:50.430 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-16T22:22:50.430 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-16T22:22:50.431 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd max markdown count = 1000 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-16T22:22:50.432 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.433 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-16T22:22:50.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-16T22:22:50.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-16T22:22:50.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-16T22:22:50.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-16T22:22:50.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-16T22:22:50.435 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:22:50.436 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-16T22:22:50.437 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-16T22:22:50.437 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-16T22:22:50.437 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-16T22:22:50.438 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-16T22:22:50.439 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-16T22:22:50.490 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.] 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC6+7JfpMSjBBAA63ePGLcFeAZZhGBEpSeszw== 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout:[client.admin] 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC6+7JfJZX6HRAAnNn60fg9K5s/8ZY6ZwnQjg== 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow *" 2020-11-16T22:22:50.505 INFO:teuthology.orchestra.run.smithi138.stdout: caps mgr = "allow *" 2020-11-16T22:22:50.506 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow *" 2020-11-16T22:22:50.506 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow *" 2020-11-16T22:22:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:22:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:22:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:206 bytes copied, 3.9591e-05 s, 5.2 MB/s 2020-11-16T22:22:50.507 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:50.507 INFO:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:22:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:22:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: �)c鬀I ���������_����_��aXa �����bXb �����cXc �����abc 2020-11-16T22:22:50.558 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:22:50.559 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:22:50.559 INFO:teuthology.orchestra.run.smithi138.stderr:409 bytes copied, 4.4761e-05 s, 9.1 MB/s 2020-11-16T22:22:50.559 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2020-11-16T22:22:50.559 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:50.560 INFO:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.560 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.610 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.623 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:22:50.624 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:22:50.624 INFO:teuthology.orchestra.run.smithi138.stderr:206 bytes copied, 3.9769e-05 s, 5.2 MB/s 2020-11-16T22:22:50.624 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.638 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:22:50.638 INFO:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.690 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.691 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:22:50.691 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:22:50.691 INFO:teuthology.orchestra.run.smithi138.stderr:409 bytes copied, 4.8667e-05 s, 8.4 MB/s 2020-11-16T22:22:50.691 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:22:50.692 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:50.692 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.692 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.707 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.722 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:22:50.722 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:22:50.722 INFO:teuthology.orchestra.run.smithi173.stderr:206 bytes copied, 2.9485e-05 s, 7.0 MB/s 2020-11-16T22:22:50.722 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.737 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:22:50.738 INFO:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.791 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.792 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:22:50.792 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:22:50.792 INFO:teuthology.orchestra.run.smithi173.stderr:409 bytes copied, 3.3175e-05 s, 12.3 MB/s 2020-11-16T22:22:50.793 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi194.front.sepia.ceph.com 2020-11-16T22:22:50.793 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:50.793 INFO:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.793 INFO:teuthology.orchestra.run.smithi194:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.808 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.823 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:22:50.823 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:22:50.823 INFO:teuthology.orchestra.run.smithi194.stderr:206 bytes copied, 3.2094e-05 s, 6.4 MB/s 2020-11-16T22:22:50.823 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.838 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:22:50.839 INFO:teuthology.orchestra.run.smithi194:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.892 INFO:teuthology.orchestra.run.smithi194.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.893 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:22:50.893 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:22:50.893 INFO:teuthology.orchestra.run.smithi194.stderr:409 bytes copied, 3.152e-05 s, 13.0 MB/s 2020-11-16T22:22:50.894 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:22:50.894 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:50.894 INFO:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.895 INFO:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.910 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:22:50.924 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:22:50.924 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:22:50.924 INFO:teuthology.orchestra.run.smithi122.stderr:206 bytes copied, 3.365e-05 s, 6.1 MB/s 2020-11-16T22:22:50.925 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:22:50.939 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:22:50.940 INFO:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.993 INFO:teuthology.orchestra.run.smithi122.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:50.994 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:22:50.994 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:22:50.994 INFO:teuthology.orchestra.run.smithi122.stderr:409 bytes copied, 3.4308e-05 s, 11.9 MB/s 2020-11-16T22:22:50.995 INFO:tasks.ceph:Setting up mon nodes... 2020-11-16T22:22:50.995 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-16T22:22:50.995 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:22:51.048 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:22:51.050 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:22:51.102 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:22:51.104 INFO:tasks.ceph:Setting up mds nodes... 2020-11-16T22:22:51.104 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:22:51.157 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:22:51.159 INFO:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-16T22:22:51.185 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-16T22:22:51.185 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-16T22:22:51.264 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-16T22:22:51.280 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-16T22:22:51.280 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2020-11-16T22:22:51.280 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-16T22:22:51.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:51.312 INFO:tasks.ceph:role: osd.0 2020-11-16T22:22:51.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:22:51.312 INFO:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:22:51.843 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:22:51.843 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:22:51.843 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-16T22:22:51.843 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:22:51.844 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:22:51.845 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2020-11-16T22:22:51.845 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-16T22:22:51.949 INFO:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-16T22:22:51.984 INFO:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:52.071 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.084+0000 7fc1ce2cbec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-16T22:22:52.074 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.087+0000 7fc1ce2cbec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-16T22:22:52.074 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.087+0000 7fc1ce2cbec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:22:52.074 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.087+0000 7fc1ce2cbec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:22:52.075 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.087+0000 7fc1ce2cbec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:22:52.079 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-16T22:22:52.093+0000 7fc1ce2cbec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-16T22:22:54.929 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-16T22:22:54.961 INFO:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-16T22:22:54.992 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:54.993 INFO:tasks.ceph:role: osd.1 2020-11-16T22:22:54.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com 2020-11-16T22:22:54.993 INFO:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:22:55.534 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:22:55.535 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-16T22:22:55.536 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:22:55.536 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:22:55.536 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:22:55.537 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2020-11-16T22:22:55.537 INFO:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2020-11-16T22:22:55.651 INFO:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-16T22:22:55.684 INFO:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:55.772 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.785+0000 7eff93dfcec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-16T22:22:55.775 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.789+0000 7eff93dfcec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-16T22:22:55.775 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.789+0000 7eff93dfcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:22:55.776 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.789+0000 7eff93dfcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:22:55.776 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.789+0000 7eff93dfcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:22:55.780 INFO:teuthology.orchestra.run.smithi194.stderr:2020-11-16T22:22:55.794+0000 7eff93dfcec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-16T22:22:58.619 INFO:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-16T22:22:58.651 INFO:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-16T22:22:58.683 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1'} 2020-11-16T22:22:58.683 INFO:tasks.ceph:role: osd.2 2020-11-16T22:22:58.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:22:58.683 INFO:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:22:59.207 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:22:59.207 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:22:59.207 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-16T22:22:59.207 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:22:59.208 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:22:59.209 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2020-11-16T22:22:59.210 INFO:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2020-11-16T22:22:59.321 INFO:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-16T22:22:59.355 INFO:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:22:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.514+0000 7f2212223ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-16T22:22:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.514+0000 7f2212223ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-16T22:22:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.514+0000 7f2212223ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:22:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.514+0000 7f2212223ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:22:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.514+0000 7f2212223ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:22:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2020-11-16T22:22:59.522+0000 7f2212223ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-16T22:23:02.351 INFO:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-16T22:23:02.383 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-16T22:23:02.383 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:23:02.383 INFO:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:23:02.399 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:23:02.414 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2020-11-16T22:23:02.414 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7Jfiz68AxAAIPflh0iIPl9KX3BIWJKsoA== 2020-11-16T22:23:02.415 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:23:02.415 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:23:02.415 INFO:teuthology.orchestra.run.smithi138.stderr:56 bytes copied, 6.303e-05 s, 888 kB/s 2020-11-16T22:23:02.416 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:23:02.416 INFO:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:23:02.470 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:23:02.484 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2020-11-16T22:23:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfO/b5BhAAIkD/vvvtCj46jDmLspWWUw== 2020-11-16T22:23:02.485 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:23:02.485 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:23:02.485 INFO:teuthology.orchestra.run.smithi138.stderr:56 bytes copied, 3.9651e-05 s, 1.4 MB/s 2020-11-16T22:23:02.486 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:23:02.486 INFO:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:23:02.540 INFO:teuthology.orchestra.run.smithi138.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:23:02.553 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2020-11-16T22:23:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfQjk8ChAAS5YbrbYos6P7eBo6fYqldg== 2020-11-16T22:23:02.554 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:23:02.554 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:23:02.554 INFO:teuthology.orchestra.run.smithi138.stderr:56 bytes copied, 5.4891e-05 s, 1.0 MB/s 2020-11-16T22:23:02.555 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:23:02.555 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:23:02.571 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:23:02.586 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2020-11-16T22:23:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQC8+7Jf+s8dBRAA5kq5a0UE7Ax59zt7Cm7V0A== 2020-11-16T22:23:02.586 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:23:02.586 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:23:02.587 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 4.8567e-05 s, 1.2 MB/s 2020-11-16T22:23:02.587 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:23:02.587 INFO:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:23:02.603 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:23:02.618 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.1] 2020-11-16T22:23:02.618 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/+7JfVcjiLhAA/ck8y05cAZLAd2im1vpiUA== 2020-11-16T22:23:02.618 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:23:02.618 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:23:02.619 INFO:teuthology.orchestra.run.smithi194.stderr:56 bytes copied, 3.8566e-05 s, 1.5 MB/s 2020-11-16T22:23:02.619 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:23:02.620 INFO:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:23:02.635 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:23:02.649 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2020-11-16T22:23:02.650 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDD+7JfJeqyHhAATKcJPfpAuHuZuiA8w363ZA== 2020-11-16T22:23:02.650 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:23:02.650 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:23:02.650 INFO:teuthology.orchestra.run.smithi122.stderr:56 bytes copied, 3.3773e-05 s, 1.7 MB/s 2020-11-16T22:23:02.651 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:23:02.651 INFO:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:23:02.665 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:23:02.665 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2020-11-16T22:23:02.666 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfA/+iEBAAiYzxcgfKj0gZTlR1Sk1Lvw== 2020-11-16T22:23:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2020-11-16T22:23:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2020-11-16T22:23:02.666 INFO:teuthology.orchestra.run.smithi138.stderr:59 bytes copied, 3.1775e-05 s, 1.9 MB/s 2020-11-16T22:23:02.666 INFO:tasks.ceph:Adding keys to all mons... 2020-11-16T22:23:02.667 INFO:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T22:23:02.732 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2020-11-16T22:23:02.732 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7Jfiz68AxAAIPflh0iIPl9KX3BIWJKsoA== 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfO/b5BhAAIkD/vvvtCj46jDmLspWWUw== 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfQjk8ChAAS5YbrbYos6P7eBo6fYqldg== 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC8+7Jf+s8dBRAA5kq5a0UE7Ax59zt7Cm7V0A== 2020-11-16T22:23:02.733 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2020-11-16T22:23:02.734 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC/+7JfVcjiLhAA/ck8y05cAZLAd2im1vpiUA== 2020-11-16T22:23:02.734 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2020-11-16T22:23:02.734 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDD+7JfJeqyHhAATKcJPfpAuHuZuiA8w363ZA== 2020-11-16T22:23:02.734 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2020-11-16T22:23:02.734 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC7+7JfA/+iEBAAiYzxcgfKj0gZTlR1Sk1Lvw== 2020-11-16T22:23:02.735 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:23:02.824 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:23:02.872 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:23:02.922 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:23:02.969 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:23:03.018 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:23:03.065 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:23:03.115 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-16T22:23:03.116 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-16T22:23:03.142 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:23:03.234 INFO:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-16T22:23:03.261 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-16T22:23:03.327 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:23:03.420 INFO:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-16T22:23:03.446 INFO:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-16T22:23:03.512 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:23:03.604 INFO:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-16T22:23:03.631 INFO:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:23:03.684 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-16T22:23:03.684 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-16T22:23:03.685 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-16T22:23:03.726 INFO:tasks.ceph.mon.a:Started 2020-11-16T22:23:03.726 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-16T22:23:03.726 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-16T22:23:03.728 INFO:tasks.ceph.mon.b:Started 2020-11-16T22:23:03.728 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-16T22:23:03.729 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-16T22:23:03.730 INFO:tasks.ceph.mon.c:Started 2020-11-16T22:23:03.731 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-16T22:23:03.731 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-16T22:23:03.734 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-16T22:23:03.735 INFO:tasks.ceph.mgr.x:Started 2020-11-16T22:23:03.736 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-16T22:23:03.736 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-16T22:23:03.738 INFO:tasks.ceph.mgr.y:Started 2020-11-16T22:23:03.738 INFO:tasks.ceph:Setting crush tunables to default 2020-11-16T22:23:03.738 INFO:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd crush tunables default 2020-11-16T22:23:04.140 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2020-11-16T22:23:04.151 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-16T22:23:04.152 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-16T22:23:04.152 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:23:04.167 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:23:04.182 INFO:teuthology.orchestra.run.smithi173.stdout:d7ac8050-ab58-4ba1-8b67-54b1751e09c0 2020-11-16T22:23:04.182 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-16T22:23:04.182 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-16T22:23:04.183 INFO:teuthology.orchestra.run.smithi173.stderr:37 bytes copied, 3.626e-05 s, 1.0 MB/s 2020-11-16T22:23:04.183 INFO:teuthology.orchestra.run.smithi194:> set -ex 2020-11-16T22:23:04.184 INFO:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:23:04.199 INFO:teuthology.orchestra.run.smithi194.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:23:04.214 INFO:teuthology.orchestra.run.smithi194.stdout:94f7af5a-9a19-4ee0-8078-95c6902e33f2 2020-11-16T22:23:04.215 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2020-11-16T22:23:04.215 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2020-11-16T22:23:04.215 INFO:teuthology.orchestra.run.smithi194.stderr:37 bytes copied, 4.5078e-05 s, 821 kB/s 2020-11-16T22:23:04.216 INFO:teuthology.orchestra.run.smithi122:> set -ex 2020-11-16T22:23:04.216 INFO:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:23:04.231 INFO:teuthology.orchestra.run.smithi122.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:23:04.246 INFO:teuthology.orchestra.run.smithi122.stdout:92637938-4ff7-497e-b599-40612e2d47dd 2020-11-16T22:23:04.246 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi122.stderr:37 bytes copied, 3.6743e-05 s, 1.0 MB/s 2020-11-16T22:23:04.247 INFO:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new d7ac8050-ab58-4ba1-8b67-54b1751e09c0 0 2020-11-16T22:23:04.524 INFO:teuthology.orchestra.run.smithi122.stdout:0 2020-11-16T22:23:04.536 INFO:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 94f7af5a-9a19-4ee0-8078-95c6902e33f2 1 2020-11-16T22:23:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:1 2020-11-16T22:23:04.759 INFO:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 92637938-4ff7-497e-b599-40612e2d47dd 2 2020-11-16T22:23:04.972 INFO:teuthology.orchestra.run.smithi122.stdout:2 2020-11-16T22:23:04.983 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-16T22:23:04.984 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-16T22:23:04.986 INFO:tasks.ceph.osd.0:Started 2020-11-16T22:23:04.986 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-16T22:23:04.987 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-16T22:23:04.989 INFO:tasks.ceph.osd.1:Started 2020-11-16T22:23:04.989 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T22:23:04.990 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T22:23:04.992 INFO:tasks.ceph.osd.2:Started 2020-11-16T22:23:04.992 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-16T22:23:05.295 INFO:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:05.488 INFO:teuthology.misc.health.smithi138.stdout: 2020-11-16T22:23:05.488 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":5,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:04.981974+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:05.499 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-11-16T22:23:05.579 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:05.593+0000 7f722596eec0 -1 Falling back to public interface 2020-11-16T22:23:05.587 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:05.600+0000 7ff60c47bec0 -1 Falling back to public interface 2020-11-16T22:23:05.588 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:05.600+0000 7f57d0fccec0 -1 Falling back to public interface 2020-11-16T22:23:06.902 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:06.916+0000 7f722596eec0 -1 osd.2 0 log_to_monitors {default=true} 2020-11-16T22:23:06.911 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:06.924+0000 7ff60c47bec0 -1 osd.0 0 log_to_monitors {default=true} 2020-11-16T22:23:06.912 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:06.924+0000 7f57d0fccec0 -1 osd.1 0 log_to_monitors {default=true} 2020-11-16T22:23:08.058 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:08.072+0000 7f721d987700 -1 osd.2 0 waiting for initial osdmap 2020-11-16T22:23:08.059 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:08.071+0000 7ff604494700 -1 osd.0 0 waiting for initial osdmap 2020-11-16T22:23:08.065 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:08.078+0000 7f57c77e2700 -1 osd.1 0 waiting for initial osdmap 2020-11-16T22:23:11.802 INFO:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:12.091 INFO:teuthology.misc.health.smithi138.stdout: 2020-11-16T22:23:12.092 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":10,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:11.084246+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:12.103 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-11-16T22:23:12.103 INFO:tasks.ceph:Creating RBD pool 2020-11-16T22:23:12.104 INFO:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-16T22:23:13.088 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'rbd' created 2020-11-16T22:23:13.099 INFO:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-16T22:23:14.094 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-16T22:23:14.105 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-16T22:23:14.105 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:23:14.106 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:23:14.108 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:23:14.108 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-16T22:23:14.109 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:14.183 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2020-11-16T22:23:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:14.403 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":12,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:14.098145+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:14.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:23:14.415 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:23:14.708 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2020-11-16T22:23:14.720 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:23:15.009 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:15.021 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:23:15.311 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:15.311 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": "device_health_metrics" 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 2, 2020-11-16T22:23:15.312 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": "rbd" 2020-11-16T22:23:15.313 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:23:15.313 INFO:teuthology.orchestra.run.smithi138.stdout:] 2020-11-16T22:23:15.323 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:23:15.324 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:23:16.113 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2020-11-16T22:23:16.124 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:23:17.122 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2020-11-16T22:23:17.134 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:23:17.495 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 3 and data pool 4 2020-11-16T22:23:17.506 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:17.793 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:17.793 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:17.499405+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:17.806 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:18.092 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:18.093 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:17.499405+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:18.105 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:23:18.514 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:18.856 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:18.857 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":4,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:23:17.499573+0000","modified":"2020-11-16T22:23:18.502988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4212":{"gid":4212,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.138:6803/1652701670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1652701670},{"type":"v1","addr":"172.21.15.138:6803","nonce":1652701670}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:23:18.858 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 4 2020-11-16T22:23:18.869 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:19.159 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:19.159 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":17,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:18.503029+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:19.172 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:19.173 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-16T22:23:19.461 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:19.462 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":17,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:18.503029+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:19.474 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:23:19.475 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:23:19.767 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2020-11-16T22:23:19.780 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:23:20.068 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:20.082 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:23:20.374 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:20.387 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:23:20.675 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:20.688 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-16T22:23:20.688 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-16T22:23:20.688 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-16T22:23:21.018 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:21.029 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":20570},{"type":"v1","addr":"172.21.15.138:6801","nonce":20570}]},"active_addr":"172.21.15.138:6801/20570","active_change":"2020-11-16T22:23:07.620934+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1284830027}]},{"addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":3811535368}]},{"addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2679916000}]}]}} 2020-11-16T22:23:21.033 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-16T22:23:21.034 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-16T22:23:21.034 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:21.324 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:21.325 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":17,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:18.503029+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:21.337 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-16T22:23:21.337 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:21.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:21.626 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":17,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:18.503029+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:21.638 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-16T22:23:21.790 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:21.791 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-16T22:23:21.941 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:21.942 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-16T22:23:22.093 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:22.093 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:23:22.386 INFO:teuthology.orchestra.run.smithi138.stdout:34359738371 2020-11-16T22:23:22.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2020-11-16T22:23:23.400 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:23:23.702 INFO:teuthology.orchestra.run.smithi138.stdout:34359738371 2020-11-16T22:23:23.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2020-11-16T22:23:24.716 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:23:25.017 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:25.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2020-11-16T22:23:25.030 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-16T22:23:25.320 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:25.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2020-11-16T22:23:25.419 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:23:25.708 INFO:teuthology.orchestra.run.smithi138.stdout:34359738372 2020-11-16T22:23:25.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.2 2020-11-16T22:23:25.720 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T22:23:25.721 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:23:26.006 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:26.007 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:23:26.018 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-11-16T22:23:25.627329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":15300,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":283100220,"statfs":{"total":289910292480,"available":289894625280,"internally_reserved":0,"allocated":503808,"data_stored":163315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.121672"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.511924+0000","last_change":"2020-11-16T22:23:17.511331+0000","last_active":"2020-11-16T22:23:17.511924+0000","last_peered":"2020-11-16T22:23:17.511924+0000","last_clean":"2020-11-16T22:23:17.511924+0000","last_became_active":"2020-11-16T22:23:17.510901+0000","last_became_peered":"2020-11-16T22:23:17.510901+0000","last_unstale":"2020-11-16T22:23:17.511924+0000","last_undegraded":"2020-11-16T22:23:17.511924+0000","last_fullsized":"2020-11-16T22:23:17.511924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509305+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:23:17.509305+0000","last_peered":"2020-11-16T22:23:17.509305+0000","last_clean":"2020-11-16T22:23:17.509305+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:23:17.509305+0000","last_undegraded":"2020-11-16T22:23:17.509305+0000","last_fullsized":"2020-11-16T22:23:17.509305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517261+0000","last_change":"2020-11-16T22:23:17.139458+0000","last_active":"2020-11-16T22:23:17.517261+0000","last_peered":"2020-11-16T22:23:17.517261+0000","last_clean":"2020-11-16T22:23:17.517261+0000","last_became_active":"2020-11-16T22:23:17.139100+0000","last_became_peered":"2020-11-16T22:23:17.139100+0000","last_unstale":"2020-11-16T22:23:17.517261+0000","last_undegraded":"2020-11-16T22:23:17.517261+0000","last_fullsized":"2020-11-16T22:23:17.517261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516571+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:23:17.516571+0000","last_peered":"2020-11-16T22:23:17.516571+0000","last_clean":"2020-11-16T22:23:17.516571+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:23:17.516571+0000","last_undegraded":"2020-11-16T22:23:17.516571+0000","last_fullsized":"2020-11-16T22:23:17.516571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506857+0000","last_change":"2020-11-16T22:23:14.110329+0000","last_active":"2020-11-16T22:23:17.506857+0000","last_peered":"2020-11-16T22:23:17.506857+0000","last_clean":"2020-11-16T22:23:17.506857+0000","last_became_active":"2020-11-16T22:23:14.110005+0000","last_became_peered":"2020-11-16T22:23:14.110005+0000","last_unstale":"2020-11-16T22:23:17.506857+0000","last_undegraded":"2020-11-16T22:23:17.506857+0000","last_fullsized":"2020-11-16T22:23:17.506857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516830+0000","last_change":"2020-11-16T22:23:17.139215+0000","last_active":"2020-11-16T22:23:17.516830+0000","last_peered":"2020-11-16T22:23:17.516830+0000","last_clean":"2020-11-16T22:23:17.516830+0000","last_became_active":"2020-11-16T22:23:17.138850+0000","last_became_peered":"2020-11-16T22:23:17.138850+0000","last_unstale":"2020-11-16T22:23:17.516830+0000","last_undegraded":"2020-11-16T22:23:17.516830+0000","last_fullsized":"2020-11-16T22:23:17.516830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516532+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:23:17.516532+0000","last_peered":"2020-11-16T22:23:17.516532+0000","last_clean":"2020-11-16T22:23:17.516532+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:23:17.516532+0000","last_undegraded":"2020-11-16T22:23:17.516532+0000","last_fullsized":"2020-11-16T22:23:17.516532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509125+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:23:17.509125+0000","last_peered":"2020-11-16T22:23:17.509125+0000","last_clean":"2020-11-16T22:23:17.509125+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:23:17.509125+0000","last_undegraded":"2020-11-16T22:23:17.509125+0000","last_fullsized":"2020-11-16T22:23:17.509125+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518398+0000","last_change":"2020-11-16T22:23:17.137728+0000","last_active":"2020-11-16T22:23:17.518398+0000","last_peered":"2020-11-16T22:23:17.518398+0000","last_clean":"2020-11-16T22:23:17.518398+0000","last_became_active":"2020-11-16T22:23:17.137370+0000","last_became_peered":"2020-11-16T22:23:17.137370+0000","last_unstale":"2020-11-16T22:23:17.518398+0000","last_undegraded":"2020-11-16T22:23:17.518398+0000","last_fullsized":"2020-11-16T22:23:17.518398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.514304+0000","last_change":"2020-11-16T22:23:17.513110+0000","last_active":"2020-11-16T22:23:17.514304+0000","last_peered":"2020-11-16T22:23:17.514304+0000","last_clean":"2020-11-16T22:23:17.514304+0000","last_became_active":"2020-11-16T22:23:17.512213+0000","last_became_peered":"2020-11-16T22:23:17.512213+0000","last_unstale":"2020-11-16T22:23:17.514304+0000","last_undegraded":"2020-11-16T22:23:17.514304+0000","last_fullsized":"2020-11-16T22:23:17.514304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506878+0000","last_change":"2020-11-16T22:23:14.110609+0000","last_active":"2020-11-16T22:23:17.506878+0000","last_peered":"2020-11-16T22:23:17.506878+0000","last_clean":"2020-11-16T22:23:17.506878+0000","last_became_active":"2020-11-16T22:23:14.110218+0000","last_became_peered":"2020-11-16T22:23:14.110218+0000","last_unstale":"2020-11-16T22:23:17.506878+0000","last_undegraded":"2020-11-16T22:23:17.506878+0000","last_fullsized":"2020-11-16T22:23:17.506878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"20","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506804+0000","last_change":"2020-11-16T22:23:10.087624+0000","last_active":"2020-11-16T22:23:17.506804+0000","last_peered":"2020-11-16T22:23:17.506804+0000","last_clean":"2020-11-16T22:23:17.506804+0000","last_became_active":"2020-11-16T22:23:10.087051+0000","last_became_peered":"2020-11-16T22:23:10.087051+0000","last_unstale":"2020-11-16T22:23:17.506804+0000","last_undegraded":"2020-11-16T22:23:17.506804+0000","last_fullsized":"2020-11-16T22:23:17.506804+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518324+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:23:17.518324+0000","last_peered":"2020-11-16T22:23:17.518324+0000","last_clean":"2020-11-16T22:23:17.518324+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:23:17.518324+0000","last_undegraded":"2020-11-16T22:23:17.518324+0000","last_fullsized":"2020-11-16T22:23:17.518324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516910+0000","last_change":"2020-11-16T22:23:17.515386+0000","last_active":"2020-11-16T22:23:17.516910+0000","last_peered":"2020-11-16T22:23:17.516910+0000","last_clean":"2020-11-16T22:23:17.516910+0000","last_became_active":"2020-11-16T22:23:17.514774+0000","last_became_peered":"2020-11-16T22:23:17.514774+0000","last_unstale":"2020-11-16T22:23:17.516910+0000","last_undegraded":"2020-11-16T22:23:17.516910+0000","last_fullsized":"2020-11-16T22:23:17.516910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506904+0000","last_change":"2020-11-16T22:23:14.107348+0000","last_active":"2020-11-16T22:23:17.506904+0000","last_peered":"2020-11-16T22:23:17.506904+0000","last_clean":"2020-11-16T22:23:17.506904+0000","last_became_active":"2020-11-16T22:23:14.106953+0000","last_became_peered":"2020-11-16T22:23:14.106953+0000","last_unstale":"2020-11-16T22:23:17.506904+0000","last_undegraded":"2020-11-16T22:23:17.506904+0000","last_fullsized":"2020-11-16T22:23:17.506904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509099+0000","last_change":"2020-11-16T22:23:17.138337+0000","last_active":"2020-11-16T22:23:17.509099+0000","last_peered":"2020-11-16T22:23:17.509099+0000","last_clean":"2020-11-16T22:23:17.509099+0000","last_became_active":"2020-11-16T22:23:17.137214+0000","last_became_peered":"2020-11-16T22:23:17.137214+0000","last_unstale":"2020-11-16T22:23:17.509099+0000","last_undegraded":"2020-11-16T22:23:17.509099+0000","last_fullsized":"2020-11-16T22:23:17.509099+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517205+0000","last_change":"2020-11-16T22:23:17.515838+0000","last_active":"2020-11-16T22:23:17.517205+0000","last_peered":"2020-11-16T22:23:17.517205+0000","last_clean":"2020-11-16T22:23:17.517205+0000","last_became_active":"2020-11-16T22:23:17.515353+0000","last_became_peered":"2020-11-16T22:23:17.515353+0000","last_unstale":"2020-11-16T22:23:17.517205+0000","last_undegraded":"2020-11-16T22:23:17.517205+0000","last_fullsized":"2020-11-16T22:23:17.517205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507535+0000","last_change":"2020-11-16T22:23:14.112895+0000","last_active":"2020-11-16T22:23:17.507535+0000","last_peered":"2020-11-16T22:23:17.507535+0000","last_clean":"2020-11-16T22:23:17.507535+0000","last_became_active":"2020-11-16T22:23:14.112524+0000","last_became_peered":"2020-11-16T22:23:14.112524+0000","last_unstale":"2020-11-16T22:23:17.507535+0000","last_undegraded":"2020-11-16T22:23:17.507535+0000","last_fullsized":"2020-11-16T22:23:17.507535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.521133+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:23:17.521133+0000","last_peered":"2020-11-16T22:23:17.521133+0000","last_clean":"2020-11-16T22:23:17.521133+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:23:17.521133+0000","last_undegraded":"2020-11-16T22:23:17.521133+0000","last_fullsized":"2020-11-16T22:23:17.521133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516050+0000","last_change":"2020-11-16T22:23:17.513578+0000","last_active":"2020-11-16T22:23:17.516050+0000","last_peered":"2020-11-16T22:23:17.516050+0000","last_clean":"2020-11-16T22:23:17.516050+0000","last_became_active":"2020-11-16T22:23:17.511994+0000","last_became_peered":"2020-11-16T22:23:17.511994+0000","last_unstale":"2020-11-16T22:23:17.516050+0000","last_undegraded":"2020-11-16T22:23:17.516050+0000","last_fullsized":"2020-11-16T22:23:17.516050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.519146+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:23:17.519146+0000","last_peered":"2020-11-16T22:23:17.519146+0000","last_clean":"2020-11-16T22:23:17.519146+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:23:17.519146+0000","last_undegraded":"2020-11-16T22:23:17.519146+0000","last_fullsized":"2020-11-16T22:23:17.519146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509414+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:23:17.509414+0000","last_peered":"2020-11-16T22:23:17.509414+0000","last_clean":"2020-11-16T22:23:17.509414+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:23:17.509414+0000","last_undegraded":"2020-11-16T22:23:17.509414+0000","last_fullsized":"2020-11-16T22:23:17.509414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.512228+0000","last_change":"2020-11-16T22:23:17.511513+0000","last_active":"2020-11-16T22:23:17.512228+0000","last_peered":"2020-11-16T22:23:17.512228+0000","last_clean":"2020-11-16T22:23:17.512228+0000","last_became_active":"2020-11-16T22:23:17.511073+0000","last_became_peered":"2020-11-16T22:23:17.511073+0000","last_unstale":"2020-11-16T22:23:17.512228+0000","last_undegraded":"2020-11-16T22:23:17.512228+0000","last_fullsized":"2020-11-16T22:23:17.512228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507024+0000","last_change":"2020-11-16T22:23:14.107363+0000","last_active":"2020-11-16T22:23:17.507024+0000","last_peered":"2020-11-16T22:23:17.507024+0000","last_clean":"2020-11-16T22:23:17.507024+0000","last_became_active":"2020-11-16T22:23:14.106918+0000","last_became_peered":"2020-11-16T22:23:14.106918+0000","last_unstale":"2020-11-16T22:23:17.507024+0000","last_undegraded":"2020-11-16T22:23:17.507024+0000","last_fullsized":"2020-11-16T22:23:17.507024+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'4","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530675+0000","last_change":"2020-11-16T22:23:17.137398+0000","last_active":"2020-11-16T22:23:18.530675+0000","last_peered":"2020-11-16T22:23:18.530675+0000","last_clean":"2020-11-16T22:23:18.530675+0000","last_became_active":"2020-11-16T22:23:17.136876+0000","last_became_peered":"2020-11-16T22:23:17.136876+0000","last_unstale":"2020-11-16T22:23:18.530675+0000","last_undegraded":"2020-11-16T22:23:18.530675+0000","last_fullsized":"2020-11-16T22:23:18.530675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5092,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366748,"statfs":{"total":96636764160,"available":96631549952,"internally_reserved":0,"allocated":159744,"data_stored":53289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":176128,"data_stored":54821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]}]}} 2020-11-16T22:23:26.020 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:23:26.307 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:26.309 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:23:26.319 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-11-16T22:23:25.627329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":15300,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":283100220,"statfs":{"total":289910292480,"available":289894625280,"internally_reserved":0,"allocated":503808,"data_stored":163315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.121672"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.511924+0000","last_change":"2020-11-16T22:23:17.511331+0000","last_active":"2020-11-16T22:23:17.511924+0000","last_peered":"2020-11-16T22:23:17.511924+0000","last_clean":"2020-11-16T22:23:17.511924+0000","last_became_active":"2020-11-16T22:23:17.510901+0000","last_became_peered":"2020-11-16T22:23:17.510901+0000","last_unstale":"2020-11-16T22:23:17.511924+0000","last_undegraded":"2020-11-16T22:23:17.511924+0000","last_fullsized":"2020-11-16T22:23:17.511924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509305+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:23:17.509305+0000","last_peered":"2020-11-16T22:23:17.509305+0000","last_clean":"2020-11-16T22:23:17.509305+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:23:17.509305+0000","last_undegraded":"2020-11-16T22:23:17.509305+0000","last_fullsized":"2020-11-16T22:23:17.509305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517261+0000","last_change":"2020-11-16T22:23:17.139458+0000","last_active":"2020-11-16T22:23:17.517261+0000","last_peered":"2020-11-16T22:23:17.517261+0000","last_clean":"2020-11-16T22:23:17.517261+0000","last_became_active":"2020-11-16T22:23:17.139100+0000","last_became_peered":"2020-11-16T22:23:17.139100+0000","last_unstale":"2020-11-16T22:23:17.517261+0000","last_undegraded":"2020-11-16T22:23:17.517261+0000","last_fullsized":"2020-11-16T22:23:17.517261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516571+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:23:17.516571+0000","last_peered":"2020-11-16T22:23:17.516571+0000","last_clean":"2020-11-16T22:23:17.516571+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:23:17.516571+0000","last_undegraded":"2020-11-16T22:23:17.516571+0000","last_fullsized":"2020-11-16T22:23:17.516571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506857+0000","last_change":"2020-11-16T22:23:14.110329+0000","last_active":"2020-11-16T22:23:17.506857+0000","last_peered":"2020-11-16T22:23:17.506857+0000","last_clean":"2020-11-16T22:23:17.506857+0000","last_became_active":"2020-11-16T22:23:14.110005+0000","last_became_peered":"2020-11-16T22:23:14.110005+0000","last_unstale":"2020-11-16T22:23:17.506857+0000","last_undegraded":"2020-11-16T22:23:17.506857+0000","last_fullsized":"2020-11-16T22:23:17.506857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516830+0000","last_change":"2020-11-16T22:23:17.139215+0000","last_active":"2020-11-16T22:23:17.516830+0000","last_peered":"2020-11-16T22:23:17.516830+0000","last_clean":"2020-11-16T22:23:17.516830+0000","last_became_active":"2020-11-16T22:23:17.138850+0000","last_became_peered":"2020-11-16T22:23:17.138850+0000","last_unstale":"2020-11-16T22:23:17.516830+0000","last_undegraded":"2020-11-16T22:23:17.516830+0000","last_fullsized":"2020-11-16T22:23:17.516830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516532+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:23:17.516532+0000","last_peered":"2020-11-16T22:23:17.516532+0000","last_clean":"2020-11-16T22:23:17.516532+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:23:17.516532+0000","last_undegraded":"2020-11-16T22:23:17.516532+0000","last_fullsized":"2020-11-16T22:23:17.516532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509125+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:23:17.509125+0000","last_peered":"2020-11-16T22:23:17.509125+0000","last_clean":"2020-11-16T22:23:17.509125+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:23:17.509125+0000","last_undegraded":"2020-11-16T22:23:17.509125+0000","last_fullsized":"2020-11-16T22:23:17.509125+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518398+0000","last_change":"2020-11-16T22:23:17.137728+0000","last_active":"2020-11-16T22:23:17.518398+0000","last_peered":"2020-11-16T22:23:17.518398+0000","last_clean":"2020-11-16T22:23:17.518398+0000","last_became_active":"2020-11-16T22:23:17.137370+0000","last_became_peered":"2020-11-16T22:23:17.137370+0000","last_unstale":"2020-11-16T22:23:17.518398+0000","last_undegraded":"2020-11-16T22:23:17.518398+0000","last_fullsized":"2020-11-16T22:23:17.518398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.514304+0000","last_change":"2020-11-16T22:23:17.513110+0000","last_active":"2020-11-16T22:23:17.514304+0000","last_peered":"2020-11-16T22:23:17.514304+0000","last_clean":"2020-11-16T22:23:17.514304+0000","last_became_active":"2020-11-16T22:23:17.512213+0000","last_became_peered":"2020-11-16T22:23:17.512213+0000","last_unstale":"2020-11-16T22:23:17.514304+0000","last_undegraded":"2020-11-16T22:23:17.514304+0000","last_fullsized":"2020-11-16T22:23:17.514304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506878+0000","last_change":"2020-11-16T22:23:14.110609+0000","last_active":"2020-11-16T22:23:17.506878+0000","last_peered":"2020-11-16T22:23:17.506878+0000","last_clean":"2020-11-16T22:23:17.506878+0000","last_became_active":"2020-11-16T22:23:14.110218+0000","last_became_peered":"2020-11-16T22:23:14.110218+0000","last_unstale":"2020-11-16T22:23:17.506878+0000","last_undegraded":"2020-11-16T22:23:17.506878+0000","last_fullsized":"2020-11-16T22:23:17.506878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"20","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506804+0000","last_change":"2020-11-16T22:23:10.087624+0000","last_active":"2020-11-16T22:23:17.506804+0000","last_peered":"2020-11-16T22:23:17.506804+0000","last_clean":"2020-11-16T22:23:17.506804+0000","last_became_active":"2020-11-16T22:23:10.087051+0000","last_became_peered":"2020-11-16T22:23:10.087051+0000","last_unstale":"2020-11-16T22:23:17.506804+0000","last_undegraded":"2020-11-16T22:23:17.506804+0000","last_fullsized":"2020-11-16T22:23:17.506804+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518324+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:23:17.518324+0000","last_peered":"2020-11-16T22:23:17.518324+0000","last_clean":"2020-11-16T22:23:17.518324+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:23:17.518324+0000","last_undegraded":"2020-11-16T22:23:17.518324+0000","last_fullsized":"2020-11-16T22:23:17.518324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516910+0000","last_change":"2020-11-16T22:23:17.515386+0000","last_active":"2020-11-16T22:23:17.516910+0000","last_peered":"2020-11-16T22:23:17.516910+0000","last_clean":"2020-11-16T22:23:17.516910+0000","last_became_active":"2020-11-16T22:23:17.514774+0000","last_became_peered":"2020-11-16T22:23:17.514774+0000","last_unstale":"2020-11-16T22:23:17.516910+0000","last_undegraded":"2020-11-16T22:23:17.516910+0000","last_fullsized":"2020-11-16T22:23:17.516910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506904+0000","last_change":"2020-11-16T22:23:14.107348+0000","last_active":"2020-11-16T22:23:17.506904+0000","last_peered":"2020-11-16T22:23:17.506904+0000","last_clean":"2020-11-16T22:23:17.506904+0000","last_became_active":"2020-11-16T22:23:14.106953+0000","last_became_peered":"2020-11-16T22:23:14.106953+0000","last_unstale":"2020-11-16T22:23:17.506904+0000","last_undegraded":"2020-11-16T22:23:17.506904+0000","last_fullsized":"2020-11-16T22:23:17.506904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509099+0000","last_change":"2020-11-16T22:23:17.138337+0000","last_active":"2020-11-16T22:23:17.509099+0000","last_peered":"2020-11-16T22:23:17.509099+0000","last_clean":"2020-11-16T22:23:17.509099+0000","last_became_active":"2020-11-16T22:23:17.137214+0000","last_became_peered":"2020-11-16T22:23:17.137214+0000","last_unstale":"2020-11-16T22:23:17.509099+0000","last_undegraded":"2020-11-16T22:23:17.509099+0000","last_fullsized":"2020-11-16T22:23:17.509099+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517205+0000","last_change":"2020-11-16T22:23:17.515838+0000","last_active":"2020-11-16T22:23:17.517205+0000","last_peered":"2020-11-16T22:23:17.517205+0000","last_clean":"2020-11-16T22:23:17.517205+0000","last_became_active":"2020-11-16T22:23:17.515353+0000","last_became_peered":"2020-11-16T22:23:17.515353+0000","last_unstale":"2020-11-16T22:23:17.517205+0000","last_undegraded":"2020-11-16T22:23:17.517205+0000","last_fullsized":"2020-11-16T22:23:17.517205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507535+0000","last_change":"2020-11-16T22:23:14.112895+0000","last_active":"2020-11-16T22:23:17.507535+0000","last_peered":"2020-11-16T22:23:17.507535+0000","last_clean":"2020-11-16T22:23:17.507535+0000","last_became_active":"2020-11-16T22:23:14.112524+0000","last_became_peered":"2020-11-16T22:23:14.112524+0000","last_unstale":"2020-11-16T22:23:17.507535+0000","last_undegraded":"2020-11-16T22:23:17.507535+0000","last_fullsized":"2020-11-16T22:23:17.507535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.521133+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:23:17.521133+0000","last_peered":"2020-11-16T22:23:17.521133+0000","last_clean":"2020-11-16T22:23:17.521133+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:23:17.521133+0000","last_undegraded":"2020-11-16T22:23:17.521133+0000","last_fullsized":"2020-11-16T22:23:17.521133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516050+0000","last_change":"2020-11-16T22:23:17.513578+0000","last_active":"2020-11-16T22:23:17.516050+0000","last_peered":"2020-11-16T22:23:17.516050+0000","last_clean":"2020-11-16T22:23:17.516050+0000","last_became_active":"2020-11-16T22:23:17.511994+0000","last_became_peered":"2020-11-16T22:23:17.511994+0000","last_unstale":"2020-11-16T22:23:17.516050+0000","last_undegraded":"2020-11-16T22:23:17.516050+0000","last_fullsized":"2020-11-16T22:23:17.516050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.519146+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:23:17.519146+0000","last_peered":"2020-11-16T22:23:17.519146+0000","last_clean":"2020-11-16T22:23:17.519146+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:23:17.519146+0000","last_undegraded":"2020-11-16T22:23:17.519146+0000","last_fullsized":"2020-11-16T22:23:17.519146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509414+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:23:17.509414+0000","last_peered":"2020-11-16T22:23:17.509414+0000","last_clean":"2020-11-16T22:23:17.509414+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:23:17.509414+0000","last_undegraded":"2020-11-16T22:23:17.509414+0000","last_fullsized":"2020-11-16T22:23:17.509414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.512228+0000","last_change":"2020-11-16T22:23:17.511513+0000","last_active":"2020-11-16T22:23:17.512228+0000","last_peered":"2020-11-16T22:23:17.512228+0000","last_clean":"2020-11-16T22:23:17.512228+0000","last_became_active":"2020-11-16T22:23:17.511073+0000","last_became_peered":"2020-11-16T22:23:17.511073+0000","last_unstale":"2020-11-16T22:23:17.512228+0000","last_undegraded":"2020-11-16T22:23:17.512228+0000","last_fullsized":"2020-11-16T22:23:17.512228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507024+0000","last_change":"2020-11-16T22:23:14.107363+0000","last_active":"2020-11-16T22:23:17.507024+0000","last_peered":"2020-11-16T22:23:17.507024+0000","last_clean":"2020-11-16T22:23:17.507024+0000","last_became_active":"2020-11-16T22:23:14.106918+0000","last_became_peered":"2020-11-16T22:23:14.106918+0000","last_unstale":"2020-11-16T22:23:17.507024+0000","last_undegraded":"2020-11-16T22:23:17.507024+0000","last_fullsized":"2020-11-16T22:23:17.507024+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'4","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530675+0000","last_change":"2020-11-16T22:23:17.137398+0000","last_active":"2020-11-16T22:23:18.530675+0000","last_peered":"2020-11-16T22:23:18.530675+0000","last_clean":"2020-11-16T22:23:18.530675+0000","last_became_active":"2020-11-16T22:23:17.136876+0000","last_became_peered":"2020-11-16T22:23:17.136876+0000","last_unstale":"2020-11-16T22:23:18.530675+0000","last_undegraded":"2020-11-16T22:23:18.530675+0000","last_fullsized":"2020-11-16T22:23:18.530675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5092,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366748,"statfs":{"total":96636764160,"available":96631549952,"internally_reserved":0,"allocated":159744,"data_stored":53289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":176128,"data_stored":54821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]}]}} 2020-11-16T22:23:26.321 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T22:23:26.321 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-16T22:23:26.321 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-16T22:23:26.322 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-16T22:23:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:26.683 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-16T22:23:26.683 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-16T22:23:26.684 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:26.972 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":17,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:18.503029+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:26.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:23:26.985 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:23:27.277 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2020-11-16T22:23:27.289 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:23:27.576 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:27.589 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:23:27.880 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:27.892 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:23:28.186 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:28.199 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:28.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:28.546 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":4,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:23:17.499573+0000","modified":"2020-11-16T22:23:18.502988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4212":{"gid":4212,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.138:6803/1652701670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1652701670},{"type":"v1","addr":"172.21.15.138:6803","nonce":1652701670}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:23:28.547 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 4 2020-11-16T22:23:28.659 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 4, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:23:17.499573+0000', 'modified': '2020-11-16T22:23:18.502988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4212}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4212': {'gid': 4212, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.138:6803/1652701670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 1652701670}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 1652701670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:23:28.659 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:23:28.660 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:23:28.798 INFO:teuthology.orchestra.run.smithi138.stdout:{"cluster_fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","whoami":0,"id":4212,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.29938845,"mdsmap_epoch":4,"osdmap_epoch":17,"osdmap_epoch_barrier":16,"uptime":14.603975417999999} 2020-11-16T22:23:28.798 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", "fs_name": "cephfs", "id": 4212, "mdsmap_epoch": 4, "osdmap_epoch": 17, "osdmap_epoch_barrier": 16, "rank_uptime": 11.29938845, "state": "up:active", "uptime": 14.603975418, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:23:28.799 INFO:teuthology.run_tasks:Running task thrashosds... 2020-11-16T22:23:28.821 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2020-11-16T22:23:28.821 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'master'}, 'ceph': {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'b637909a9153a419511f943e4eee7800ad3bbb99'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'master', 'sha1': '6441771be52a0f47b9e118f8bca335f29dfda438'}} 2020-11-16T22:23:28.822 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2020-11-16T22:23:28.822 INFO:tasks.thrashosds:Doing preliminary sync to avoid collateral damage... 2020-11-16T22:23:28.822 INFO:teuthology.orchestra.run.smithi122:> sync 2020-11-16T22:23:28.997 INFO:teuthology.orchestra.run.smithi138:> sync 2020-11-16T22:23:29.160 INFO:teuthology.orchestra.run.smithi173:> sync 2020-11-16T22:23:29.279 INFO:teuthology.orchestra.run.smithi194:> sync 2020-11-16T22:23:29.429 DEBUG:tasks.thrashosds:checking console status of smithi138 2020-11-16T22:23:29.429 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi138 2020-11-16T22:23:29.430 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi138 2020-11-16T22:23:29.491 DEBUG:teuthology.orchestra.console:expect: smithi138 login 2020-11-16T22:23:29.681 DEBUG:teuthology.orchestra.console:expect before: b"\r\n[Enter `^Ec?' for help]\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n" 2020-11-16T22:23:29.681 DEBUG:teuthology.orchestra.console:expect after: b'smithi138 login: ' 2020-11-16T22:23:29.835 DEBUG:tasks.thrashosds:checking console status of smithi173 2020-11-16T22:23:29.836 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi173 2020-11-16T22:23:29.836 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi173 2020-11-16T22:23:29.895 DEBUG:teuthology.orchestra.console:expect: smithi173 login 2020-11-16T22:23:30.150 DEBUG:teuthology.orchestra.console:expect before: b"\r\n[Enter `^Ec?' for help]\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n" 2020-11-16T22:23:30.150 DEBUG:teuthology.orchestra.console:expect after: b'smithi173 login: ' 2020-11-16T22:23:30.305 DEBUG:tasks.thrashosds:checking console status of smithi194 2020-11-16T22:23:30.306 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi194 2020-11-16T22:23:30.306 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi194 2020-11-16T22:23:30.365 DEBUG:teuthology.orchestra.console:expect: smithi194 login 2020-11-16T22:23:30.564 DEBUG:teuthology.orchestra.console:expect before: b"\r\n[Enter `^Ec?' for help]\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n" 2020-11-16T22:23:30.564 DEBUG:teuthology.orchestra.console:expect after: b'smithi194 login: ' 2020-11-16T22:23:30.719 DEBUG:tasks.thrashosds:checking console status of smithi122 2020-11-16T22:23:30.719 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi122 2020-11-16T22:23:30.720 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi122 2020-11-16T22:23:30.779 DEBUG:teuthology.orchestra.console:expect: smithi122 login 2020-11-16T22:23:30.950 DEBUG:teuthology.orchestra.console:expect before: b"\r\n[Enter `^Ec?' for help]\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n" 2020-11-16T22:23:30.951 DEBUG:teuthology.orchestra.console:expect after: b'smithi122 login: ' 2020-11-16T22:23:31.103 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2020-11-16T22:23:31.423 INFO:tasks.thrashosds:Beginning thrashosds... 2020-11-16T22:23:31.424 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T22:23:31.424 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:23:31.711 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:31.712 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:23:31.723 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2020-11-16T22:23:31.628391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":15300,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":283100220,"statfs":{"total":289910292480,"available":289894625280,"internally_reserved":0,"allocated":503808,"data_stored":163315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002089"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.511924+0000","last_change":"2020-11-16T22:23:17.511331+0000","last_active":"2020-11-16T22:23:17.511924+0000","last_peered":"2020-11-16T22:23:17.511924+0000","last_clean":"2020-11-16T22:23:17.511924+0000","last_became_active":"2020-11-16T22:23:17.510901+0000","last_became_peered":"2020-11-16T22:23:17.510901+0000","last_unstale":"2020-11-16T22:23:17.511924+0000","last_undegraded":"2020-11-16T22:23:17.511924+0000","last_fullsized":"2020-11-16T22:23:17.511924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509305+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:23:17.509305+0000","last_peered":"2020-11-16T22:23:17.509305+0000","last_clean":"2020-11-16T22:23:17.509305+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:23:17.509305+0000","last_undegraded":"2020-11-16T22:23:17.509305+0000","last_fullsized":"2020-11-16T22:23:17.509305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517261+0000","last_change":"2020-11-16T22:23:17.139458+0000","last_active":"2020-11-16T22:23:17.517261+0000","last_peered":"2020-11-16T22:23:17.517261+0000","last_clean":"2020-11-16T22:23:17.517261+0000","last_became_active":"2020-11-16T22:23:17.139100+0000","last_became_peered":"2020-11-16T22:23:17.139100+0000","last_unstale":"2020-11-16T22:23:17.517261+0000","last_undegraded":"2020-11-16T22:23:17.517261+0000","last_fullsized":"2020-11-16T22:23:17.517261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516571+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:23:17.516571+0000","last_peered":"2020-11-16T22:23:17.516571+0000","last_clean":"2020-11-16T22:23:17.516571+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:23:17.516571+0000","last_undegraded":"2020-11-16T22:23:17.516571+0000","last_fullsized":"2020-11-16T22:23:17.516571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506857+0000","last_change":"2020-11-16T22:23:14.110329+0000","last_active":"2020-11-16T22:23:17.506857+0000","last_peered":"2020-11-16T22:23:17.506857+0000","last_clean":"2020-11-16T22:23:17.506857+0000","last_became_active":"2020-11-16T22:23:14.110005+0000","last_became_peered":"2020-11-16T22:23:14.110005+0000","last_unstale":"2020-11-16T22:23:17.506857+0000","last_undegraded":"2020-11-16T22:23:17.506857+0000","last_fullsized":"2020-11-16T22:23:17.506857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516830+0000","last_change":"2020-11-16T22:23:17.139215+0000","last_active":"2020-11-16T22:23:17.516830+0000","last_peered":"2020-11-16T22:23:17.516830+0000","last_clean":"2020-11-16T22:23:17.516830+0000","last_became_active":"2020-11-16T22:23:17.138850+0000","last_became_peered":"2020-11-16T22:23:17.138850+0000","last_unstale":"2020-11-16T22:23:17.516830+0000","last_undegraded":"2020-11-16T22:23:17.516830+0000","last_fullsized":"2020-11-16T22:23:17.516830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516532+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:23:17.516532+0000","last_peered":"2020-11-16T22:23:17.516532+0000","last_clean":"2020-11-16T22:23:17.516532+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:23:17.516532+0000","last_undegraded":"2020-11-16T22:23:17.516532+0000","last_fullsized":"2020-11-16T22:23:17.516532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509125+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:23:17.509125+0000","last_peered":"2020-11-16T22:23:17.509125+0000","last_clean":"2020-11-16T22:23:17.509125+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:23:17.509125+0000","last_undegraded":"2020-11-16T22:23:17.509125+0000","last_fullsized":"2020-11-16T22:23:17.509125+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518398+0000","last_change":"2020-11-16T22:23:17.137728+0000","last_active":"2020-11-16T22:23:17.518398+0000","last_peered":"2020-11-16T22:23:17.518398+0000","last_clean":"2020-11-16T22:23:17.518398+0000","last_became_active":"2020-11-16T22:23:17.137370+0000","last_became_peered":"2020-11-16T22:23:17.137370+0000","last_unstale":"2020-11-16T22:23:17.518398+0000","last_undegraded":"2020-11-16T22:23:17.518398+0000","last_fullsized":"2020-11-16T22:23:17.518398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.514304+0000","last_change":"2020-11-16T22:23:17.513110+0000","last_active":"2020-11-16T22:23:17.514304+0000","last_peered":"2020-11-16T22:23:17.514304+0000","last_clean":"2020-11-16T22:23:17.514304+0000","last_became_active":"2020-11-16T22:23:17.512213+0000","last_became_peered":"2020-11-16T22:23:17.512213+0000","last_unstale":"2020-11-16T22:23:17.514304+0000","last_undegraded":"2020-11-16T22:23:17.514304+0000","last_fullsized":"2020-11-16T22:23:17.514304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506878+0000","last_change":"2020-11-16T22:23:14.110609+0000","last_active":"2020-11-16T22:23:17.506878+0000","last_peered":"2020-11-16T22:23:17.506878+0000","last_clean":"2020-11-16T22:23:17.506878+0000","last_became_active":"2020-11-16T22:23:14.110218+0000","last_became_peered":"2020-11-16T22:23:14.110218+0000","last_unstale":"2020-11-16T22:23:17.506878+0000","last_undegraded":"2020-11-16T22:23:17.506878+0000","last_fullsized":"2020-11-16T22:23:17.506878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"20","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506804+0000","last_change":"2020-11-16T22:23:10.087624+0000","last_active":"2020-11-16T22:23:17.506804+0000","last_peered":"2020-11-16T22:23:17.506804+0000","last_clean":"2020-11-16T22:23:17.506804+0000","last_became_active":"2020-11-16T22:23:10.087051+0000","last_became_peered":"2020-11-16T22:23:10.087051+0000","last_unstale":"2020-11-16T22:23:17.506804+0000","last_undegraded":"2020-11-16T22:23:17.506804+0000","last_fullsized":"2020-11-16T22:23:17.506804+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518324+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:23:17.518324+0000","last_peered":"2020-11-16T22:23:17.518324+0000","last_clean":"2020-11-16T22:23:17.518324+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:23:17.518324+0000","last_undegraded":"2020-11-16T22:23:17.518324+0000","last_fullsized":"2020-11-16T22:23:17.518324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516910+0000","last_change":"2020-11-16T22:23:17.515386+0000","last_active":"2020-11-16T22:23:17.516910+0000","last_peered":"2020-11-16T22:23:17.516910+0000","last_clean":"2020-11-16T22:23:17.516910+0000","last_became_active":"2020-11-16T22:23:17.514774+0000","last_became_peered":"2020-11-16T22:23:17.514774+0000","last_unstale":"2020-11-16T22:23:17.516910+0000","last_undegraded":"2020-11-16T22:23:17.516910+0000","last_fullsized":"2020-11-16T22:23:17.516910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506904+0000","last_change":"2020-11-16T22:23:14.107348+0000","last_active":"2020-11-16T22:23:17.506904+0000","last_peered":"2020-11-16T22:23:17.506904+0000","last_clean":"2020-11-16T22:23:17.506904+0000","last_became_active":"2020-11-16T22:23:14.106953+0000","last_became_peered":"2020-11-16T22:23:14.106953+0000","last_unstale":"2020-11-16T22:23:17.506904+0000","last_undegraded":"2020-11-16T22:23:17.506904+0000","last_fullsized":"2020-11-16T22:23:17.506904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509099+0000","last_change":"2020-11-16T22:23:17.138337+0000","last_active":"2020-11-16T22:23:17.509099+0000","last_peered":"2020-11-16T22:23:17.509099+0000","last_clean":"2020-11-16T22:23:17.509099+0000","last_became_active":"2020-11-16T22:23:17.137214+0000","last_became_peered":"2020-11-16T22:23:17.137214+0000","last_unstale":"2020-11-16T22:23:17.509099+0000","last_undegraded":"2020-11-16T22:23:17.509099+0000","last_fullsized":"2020-11-16T22:23:17.509099+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517205+0000","last_change":"2020-11-16T22:23:17.515838+0000","last_active":"2020-11-16T22:23:17.517205+0000","last_peered":"2020-11-16T22:23:17.517205+0000","last_clean":"2020-11-16T22:23:17.517205+0000","last_became_active":"2020-11-16T22:23:17.515353+0000","last_became_peered":"2020-11-16T22:23:17.515353+0000","last_unstale":"2020-11-16T22:23:17.517205+0000","last_undegraded":"2020-11-16T22:23:17.517205+0000","last_fullsized":"2020-11-16T22:23:17.517205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507535+0000","last_change":"2020-11-16T22:23:14.112895+0000","last_active":"2020-11-16T22:23:17.507535+0000","last_peered":"2020-11-16T22:23:17.507535+0000","last_clean":"2020-11-16T22:23:17.507535+0000","last_became_active":"2020-11-16T22:23:14.112524+0000","last_became_peered":"2020-11-16T22:23:14.112524+0000","last_unstale":"2020-11-16T22:23:17.507535+0000","last_undegraded":"2020-11-16T22:23:17.507535+0000","last_fullsized":"2020-11-16T22:23:17.507535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.521133+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:23:17.521133+0000","last_peered":"2020-11-16T22:23:17.521133+0000","last_clean":"2020-11-16T22:23:17.521133+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:23:17.521133+0000","last_undegraded":"2020-11-16T22:23:17.521133+0000","last_fullsized":"2020-11-16T22:23:17.521133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516050+0000","last_change":"2020-11-16T22:23:17.513578+0000","last_active":"2020-11-16T22:23:17.516050+0000","last_peered":"2020-11-16T22:23:17.516050+0000","last_clean":"2020-11-16T22:23:17.516050+0000","last_became_active":"2020-11-16T22:23:17.511994+0000","last_became_peered":"2020-11-16T22:23:17.511994+0000","last_unstale":"2020-11-16T22:23:17.516050+0000","last_undegraded":"2020-11-16T22:23:17.516050+0000","last_fullsized":"2020-11-16T22:23:17.516050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.519146+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:23:17.519146+0000","last_peered":"2020-11-16T22:23:17.519146+0000","last_clean":"2020-11-16T22:23:17.519146+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:23:17.519146+0000","last_undegraded":"2020-11-16T22:23:17.519146+0000","last_fullsized":"2020-11-16T22:23:17.519146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509414+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:23:17.509414+0000","last_peered":"2020-11-16T22:23:17.509414+0000","last_clean":"2020-11-16T22:23:17.509414+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:23:17.509414+0000","last_undegraded":"2020-11-16T22:23:17.509414+0000","last_fullsized":"2020-11-16T22:23:17.509414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.512228+0000","last_change":"2020-11-16T22:23:17.511513+0000","last_active":"2020-11-16T22:23:17.512228+0000","last_peered":"2020-11-16T22:23:17.512228+0000","last_clean":"2020-11-16T22:23:17.512228+0000","last_became_active":"2020-11-16T22:23:17.511073+0000","last_became_peered":"2020-11-16T22:23:17.511073+0000","last_unstale":"2020-11-16T22:23:17.512228+0000","last_undegraded":"2020-11-16T22:23:17.512228+0000","last_fullsized":"2020-11-16T22:23:17.512228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507024+0000","last_change":"2020-11-16T22:23:14.107363+0000","last_active":"2020-11-16T22:23:17.507024+0000","last_peered":"2020-11-16T22:23:17.507024+0000","last_clean":"2020-11-16T22:23:17.507024+0000","last_became_active":"2020-11-16T22:23:14.106918+0000","last_became_peered":"2020-11-16T22:23:14.106918+0000","last_unstale":"2020-11-16T22:23:17.507024+0000","last_undegraded":"2020-11-16T22:23:17.507024+0000","last_fullsized":"2020-11-16T22:23:17.507024+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'4","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530675+0000","last_change":"2020-11-16T22:23:17.137398+0000","last_active":"2020-11-16T22:23:18.530675+0000","last_peered":"2020-11-16T22:23:18.530675+0000","last_clean":"2020-11-16T22:23:18.530675+0000","last_became_active":"2020-11-16T22:23:17.136876+0000","last_became_peered":"2020-11-16T22:23:17.136876+0000","last_unstale":"2020-11-16T22:23:18.530675+0000","last_undegraded":"2020-11-16T22:23:18.530675+0000","last_fullsized":"2020-11-16T22:23:18.530675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5092,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366748,"statfs":{"total":96636764160,"available":96631549952,"internally_reserved":0,"allocated":159744,"data_stored":53289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":176128,"data_stored":54821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]}]}} 2020-11-16T22:23:31.725 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:23:32.011 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:32.012 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:23:32.023 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2020-11-16T22:23:31.628391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":15300,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":283100220,"statfs":{"total":289910292480,"available":289894625280,"internally_reserved":0,"allocated":503808,"data_stored":163315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002089"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.511924+0000","last_change":"2020-11-16T22:23:17.511331+0000","last_active":"2020-11-16T22:23:17.511924+0000","last_peered":"2020-11-16T22:23:17.511924+0000","last_clean":"2020-11-16T22:23:17.511924+0000","last_became_active":"2020-11-16T22:23:17.510901+0000","last_became_peered":"2020-11-16T22:23:17.510901+0000","last_unstale":"2020-11-16T22:23:17.511924+0000","last_undegraded":"2020-11-16T22:23:17.511924+0000","last_fullsized":"2020-11-16T22:23:17.511924+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509305+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:23:17.509305+0000","last_peered":"2020-11-16T22:23:17.509305+0000","last_clean":"2020-11-16T22:23:17.509305+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:23:17.509305+0000","last_undegraded":"2020-11-16T22:23:17.509305+0000","last_fullsized":"2020-11-16T22:23:17.509305+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517261+0000","last_change":"2020-11-16T22:23:17.139458+0000","last_active":"2020-11-16T22:23:17.517261+0000","last_peered":"2020-11-16T22:23:17.517261+0000","last_clean":"2020-11-16T22:23:17.517261+0000","last_became_active":"2020-11-16T22:23:17.139100+0000","last_became_peered":"2020-11-16T22:23:17.139100+0000","last_unstale":"2020-11-16T22:23:17.517261+0000","last_undegraded":"2020-11-16T22:23:17.517261+0000","last_fullsized":"2020-11-16T22:23:17.517261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516571+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:23:17.516571+0000","last_peered":"2020-11-16T22:23:17.516571+0000","last_clean":"2020-11-16T22:23:17.516571+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:23:17.516571+0000","last_undegraded":"2020-11-16T22:23:17.516571+0000","last_fullsized":"2020-11-16T22:23:17.516571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506857+0000","last_change":"2020-11-16T22:23:14.110329+0000","last_active":"2020-11-16T22:23:17.506857+0000","last_peered":"2020-11-16T22:23:17.506857+0000","last_clean":"2020-11-16T22:23:17.506857+0000","last_became_active":"2020-11-16T22:23:14.110005+0000","last_became_peered":"2020-11-16T22:23:14.110005+0000","last_unstale":"2020-11-16T22:23:17.506857+0000","last_undegraded":"2020-11-16T22:23:17.506857+0000","last_fullsized":"2020-11-16T22:23:17.506857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516830+0000","last_change":"2020-11-16T22:23:17.139215+0000","last_active":"2020-11-16T22:23:17.516830+0000","last_peered":"2020-11-16T22:23:17.516830+0000","last_clean":"2020-11-16T22:23:17.516830+0000","last_became_active":"2020-11-16T22:23:17.138850+0000","last_became_peered":"2020-11-16T22:23:17.138850+0000","last_unstale":"2020-11-16T22:23:17.516830+0000","last_undegraded":"2020-11-16T22:23:17.516830+0000","last_fullsized":"2020-11-16T22:23:17.516830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516532+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:23:17.516532+0000","last_peered":"2020-11-16T22:23:17.516532+0000","last_clean":"2020-11-16T22:23:17.516532+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:23:17.516532+0000","last_undegraded":"2020-11-16T22:23:17.516532+0000","last_fullsized":"2020-11-16T22:23:17.516532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509125+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:23:17.509125+0000","last_peered":"2020-11-16T22:23:17.509125+0000","last_clean":"2020-11-16T22:23:17.509125+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:23:17.509125+0000","last_undegraded":"2020-11-16T22:23:17.509125+0000","last_fullsized":"2020-11-16T22:23:17.509125+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518398+0000","last_change":"2020-11-16T22:23:17.137728+0000","last_active":"2020-11-16T22:23:17.518398+0000","last_peered":"2020-11-16T22:23:17.518398+0000","last_clean":"2020-11-16T22:23:17.518398+0000","last_became_active":"2020-11-16T22:23:17.137370+0000","last_became_peered":"2020-11-16T22:23:17.137370+0000","last_unstale":"2020-11-16T22:23:17.518398+0000","last_undegraded":"2020-11-16T22:23:17.518398+0000","last_fullsized":"2020-11-16T22:23:17.518398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.514304+0000","last_change":"2020-11-16T22:23:17.513110+0000","last_active":"2020-11-16T22:23:17.514304+0000","last_peered":"2020-11-16T22:23:17.514304+0000","last_clean":"2020-11-16T22:23:17.514304+0000","last_became_active":"2020-11-16T22:23:17.512213+0000","last_became_peered":"2020-11-16T22:23:17.512213+0000","last_unstale":"2020-11-16T22:23:17.514304+0000","last_undegraded":"2020-11-16T22:23:17.514304+0000","last_fullsized":"2020-11-16T22:23:17.514304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506878+0000","last_change":"2020-11-16T22:23:14.110609+0000","last_active":"2020-11-16T22:23:17.506878+0000","last_peered":"2020-11-16T22:23:17.506878+0000","last_clean":"2020-11-16T22:23:17.506878+0000","last_became_active":"2020-11-16T22:23:14.110218+0000","last_became_peered":"2020-11-16T22:23:14.110218+0000","last_unstale":"2020-11-16T22:23:17.506878+0000","last_undegraded":"2020-11-16T22:23:17.506878+0000","last_fullsized":"2020-11-16T22:23:17.506878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"20","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506804+0000","last_change":"2020-11-16T22:23:10.087624+0000","last_active":"2020-11-16T22:23:17.506804+0000","last_peered":"2020-11-16T22:23:17.506804+0000","last_clean":"2020-11-16T22:23:17.506804+0000","last_became_active":"2020-11-16T22:23:10.087051+0000","last_became_peered":"2020-11-16T22:23:10.087051+0000","last_unstale":"2020-11-16T22:23:17.506804+0000","last_undegraded":"2020-11-16T22:23:17.506804+0000","last_fullsized":"2020-11-16T22:23:17.506804+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.518324+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:23:17.518324+0000","last_peered":"2020-11-16T22:23:17.518324+0000","last_clean":"2020-11-16T22:23:17.518324+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:23:17.518324+0000","last_undegraded":"2020-11-16T22:23:17.518324+0000","last_fullsized":"2020-11-16T22:23:17.518324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516910+0000","last_change":"2020-11-16T22:23:17.515386+0000","last_active":"2020-11-16T22:23:17.516910+0000","last_peered":"2020-11-16T22:23:17.516910+0000","last_clean":"2020-11-16T22:23:17.516910+0000","last_became_active":"2020-11-16T22:23:17.514774+0000","last_became_peered":"2020-11-16T22:23:17.514774+0000","last_unstale":"2020-11-16T22:23:17.516910+0000","last_undegraded":"2020-11-16T22:23:17.516910+0000","last_fullsized":"2020-11-16T22:23:17.516910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.506904+0000","last_change":"2020-11-16T22:23:14.107348+0000","last_active":"2020-11-16T22:23:17.506904+0000","last_peered":"2020-11-16T22:23:17.506904+0000","last_clean":"2020-11-16T22:23:17.506904+0000","last_became_active":"2020-11-16T22:23:14.106953+0000","last_became_peered":"2020-11-16T22:23:14.106953+0000","last_unstale":"2020-11-16T22:23:17.506904+0000","last_undegraded":"2020-11-16T22:23:17.506904+0000","last_fullsized":"2020-11-16T22:23:17.506904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509099+0000","last_change":"2020-11-16T22:23:17.138337+0000","last_active":"2020-11-16T22:23:17.509099+0000","last_peered":"2020-11-16T22:23:17.509099+0000","last_clean":"2020-11-16T22:23:17.509099+0000","last_became_active":"2020-11-16T22:23:17.137214+0000","last_became_peered":"2020-11-16T22:23:17.137214+0000","last_unstale":"2020-11-16T22:23:17.509099+0000","last_undegraded":"2020-11-16T22:23:17.509099+0000","last_fullsized":"2020-11-16T22:23:17.509099+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.517205+0000","last_change":"2020-11-16T22:23:17.515838+0000","last_active":"2020-11-16T22:23:17.517205+0000","last_peered":"2020-11-16T22:23:17.517205+0000","last_clean":"2020-11-16T22:23:17.517205+0000","last_became_active":"2020-11-16T22:23:17.515353+0000","last_became_peered":"2020-11-16T22:23:17.515353+0000","last_unstale":"2020-11-16T22:23:17.517205+0000","last_undegraded":"2020-11-16T22:23:17.517205+0000","last_fullsized":"2020-11-16T22:23:17.517205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507535+0000","last_change":"2020-11-16T22:23:14.112895+0000","last_active":"2020-11-16T22:23:17.507535+0000","last_peered":"2020-11-16T22:23:17.507535+0000","last_clean":"2020-11-16T22:23:17.507535+0000","last_became_active":"2020-11-16T22:23:14.112524+0000","last_became_peered":"2020-11-16T22:23:14.112524+0000","last_unstale":"2020-11-16T22:23:17.507535+0000","last_undegraded":"2020-11-16T22:23:17.507535+0000","last_fullsized":"2020-11-16T22:23:17.507535+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.521133+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:23:17.521133+0000","last_peered":"2020-11-16T22:23:17.521133+0000","last_clean":"2020-11-16T22:23:17.521133+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:23:17.521133+0000","last_undegraded":"2020-11-16T22:23:17.521133+0000","last_fullsized":"2020-11-16T22:23:17.521133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.516050+0000","last_change":"2020-11-16T22:23:17.513578+0000","last_active":"2020-11-16T22:23:17.516050+0000","last_peered":"2020-11-16T22:23:17.516050+0000","last_clean":"2020-11-16T22:23:17.516050+0000","last_became_active":"2020-11-16T22:23:17.511994+0000","last_became_peered":"2020-11-16T22:23:17.511994+0000","last_unstale":"2020-11-16T22:23:17.516050+0000","last_undegraded":"2020-11-16T22:23:17.516050+0000","last_fullsized":"2020-11-16T22:23:17.516050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.519146+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:23:17.519146+0000","last_peered":"2020-11-16T22:23:17.519146+0000","last_clean":"2020-11-16T22:23:17.519146+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:23:17.519146+0000","last_undegraded":"2020-11-16T22:23:17.519146+0000","last_fullsized":"2020-11-16T22:23:17.519146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.509414+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:23:17.509414+0000","last_peered":"2020-11-16T22:23:17.509414+0000","last_clean":"2020-11-16T22:23:17.509414+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:23:17.509414+0000","last_undegraded":"2020-11-16T22:23:17.509414+0000","last_fullsized":"2020-11-16T22:23:17.509414+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.512228+0000","last_change":"2020-11-16T22:23:17.511513+0000","last_active":"2020-11-16T22:23:17.512228+0000","last_peered":"2020-11-16T22:23:17.512228+0000","last_clean":"2020-11-16T22:23:17.512228+0000","last_became_active":"2020-11-16T22:23:17.511073+0000","last_became_peered":"2020-11-16T22:23:17.511073+0000","last_unstale":"2020-11-16T22:23:17.512228+0000","last_undegraded":"2020-11-16T22:23:17.512228+0000","last_fullsized":"2020-11-16T22:23:17.512228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-11-16T22:23:17.507024+0000","last_change":"2020-11-16T22:23:14.107363+0000","last_active":"2020-11-16T22:23:17.507024+0000","last_peered":"2020-11-16T22:23:17.507024+0000","last_clean":"2020-11-16T22:23:17.507024+0000","last_became_active":"2020-11-16T22:23:14.106918+0000","last_became_peered":"2020-11-16T22:23:14.106918+0000","last_unstale":"2020-11-16T22:23:17.507024+0000","last_undegraded":"2020-11-16T22:23:17.507024+0000","last_fullsized":"2020-11-16T22:23:17.507024+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'4","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530675+0000","last_change":"2020-11-16T22:23:17.137398+0000","last_active":"2020-11-16T22:23:18.530675+0000","last_peered":"2020-11-16T22:23:18.530675+0000","last_clean":"2020-11-16T22:23:18.530675+0000","last_became_active":"2020-11-16T22:23:17.136876+0000","last_became_peered":"2020-11-16T22:23:17.136876+0000","last_unstale":"2020-11-16T22:23:18.530675+0000","last_undegraded":"2020-11-16T22:23:18.530675+0000","last_fullsized":"2020-11-16T22:23:18.530675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5092,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366748,"statfs":{"total":96636764160,"available":96631549952,"internally_reserved":0,"allocated":159744,"data_stored":53289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":176128,"data_stored":54821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]}]}} 2020-11-16T22:23:32.025 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T22:23:32.025 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2020-11-16T22:23:32.310 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 17 2020-11-16T22:23:32.310 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 2963e9ac-8003-490b-a7b3-aee3b9109ae1 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:created 2020-11-16T22:23:04.049027+0000 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:modified 2020-11-16T22:23:18.503029+0000 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:crush_version 4 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:full_ratio 0.95 2020-11-16T22:23:32.311 INFO:teuthology.orchestra.run.smithi138.stdout:backfillfull_ratio 0.9 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:nearfull_ratio 0.85 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:require_min_compat_client luminous 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:min_compat_client jewel 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:require_osd_release pacific 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:stretch_mode_enabled false 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:pool 1 'device_health_metrics' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 10 flags hashpspool stripe_width 0 pg_num_min 1 application mgr_devicehealth 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool stripe_width 0 application rbd 2020-11-16T22:23:32.312 INFO:teuthology.orchestra.run.smithi138.stdout:pool 3 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs 2020-11-16T22:23:32.313 INFO:teuthology.orchestra.run.smithi138.stdout:pool 4 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application cephfs 2020-11-16T22:23:32.313 INFO:teuthology.orchestra.run.smithi138.stdout:max_osd 3 2020-11-16T22:23:32.313 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/20752,v1:172.21.15.173:6801/20752] [v2:172.21.15.173:6802/20752,v1:172.21.15.173:6803/20752] exists,up d7ac8050-ab58-4ba1-8b67-54b1751e09c0 2020-11-16T22:23:32.313 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/20794,v1:172.21.15.194:6801/20794] [v2:172.21.15.194:6802/20794,v1:172.21.15.194:6803/20794] exists,up 94f7af5a-9a19-4ee0-8078-95c6902e33f2 2020-11-16T22:23:32.313 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/20834,v1:172.21.15.122:6801/20834] [v2:172.21.15.122:6802/20834,v1:172.21.15.122:6803/20834] exists,up 92637938-4ff7-497e-b599-40612e2d47dd 2020-11-16T22:23:32.322 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/20752,v1:172.21.15.173:6801/20752] [v2:172.21.15.173:6802/20752,v1:172.21.15.173:6803/20752] exists,up d7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'osd.1 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/20794,v1:172.21.15.194:6801/20794] [v2:172.21.15.194:6802/20794,v1:172.21.15.194:6803/20794] exists,up 94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'osd.2 up in weight 1 up_from 8 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/20834,v1:172.21.15.122:6801/20834] [v2:172.21.15.122:6802/20834,v1:172.21.15.122:6803/20834] exists,up 92637938-4ff7-497e-b599-40612e2d47dd'] 2020-11-16T22:23:32.323 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2020-11-16T22:23:32.443 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:23:32.443 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mon.a", 2020-11-16T22:23:32.443 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster": "ceph", 2020-11-16T22:23:32.443 INFO:teuthology.orchestra.run.smithi138.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2020-11-16T22:23:32.443 INFO:teuthology.orchestra.run.smithi138.stdout: "admin_socket_mode": "", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_client_required": "cephx, none", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_cluster_required": "cephx", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_debug": "true", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_mon_ticket_ttl": "43200.000000", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_service_required": "cephx", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_service_ticket_ttl": "3600.000000", 2020-11-16T22:23:32.444 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_supported": "cephx", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio": "true", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_max_queue_depth": "1024", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_poll_ms": "250", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_aio_reap_max": "16", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_async_discard": "false", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_block_size": "4096", 2020-11-16T22:23:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio": "false", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio_log_age": "5.000000", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_debug_inflight_ios": "false", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_enable_discard": "false", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_flock_retry": "3", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_flock_retry_interval": "0.100000", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_inject_crash": "0", 2020-11-16T22:23:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_inject_crash_flush_delay": "2", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_ioring": "false", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_nvme_unbind_from_kernel": "false", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bdev_type": "", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_alloc_size": "1048576", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_allocator": "hybrid", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_buffered_io": "false", 2020-11-16T22:23:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_compact_log_sync": "false", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_compact_min_size": "16777216", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_log_replay_check_allocations": "true", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_max_log_runway": "4194304", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_max_prefetch": "1048576", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_min_flush_size": "524288", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_min_log_runway": "1048576", 2020-11-16T22:23:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_replay_recovery": "false", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_replay_recovery_disable_compact": "false", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_shared_alloc_size": "65536", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_sync_write": "false", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2020-11-16T22:23:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_allocator": "hybrid", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bitmapallocator_span_size": "1024", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blobid_prealloc": "10240", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_create": "true", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_create": "false", 2020-11-16T22:23:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_path": "", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_db_size": "0", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_path": "", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_preallocate_file": "false", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_size": "107374182400", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_create": "false", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_path": "", 2020-11-16T22:23:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_block_wal_size": "100663296", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs": "true", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_bluefs_env_mirror": "false", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_autotune": "true", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_autotune_interval": "5.000000", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_kv_ratio": "0.400000", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_ratio": "0.400000", 2020-11-16T22:23:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size": "0", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size_hdd": "1073741824", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_size_ssd": "3221225472", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_trim_interval": "0.050000", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_type": "2q", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_clone_cow": "true", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_algorithm": "snappy", 2020-11-16T22:23:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size": "0", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size": "0", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_mode": "none", 2020-11-16T22:23:32.454 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_compression_required_ratio": "0.875000", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_csum_type": "crc32c", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_enforce_settings": "default", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_freelist": "false", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_fsck_abort": "false", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_bug21040": "false", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_inject_read_err": "false", 2020-11-16T22:23:32.455 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_no_reuse_blocks": "false", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_omit_block_device_write": "false", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_omit_kv_commit": "false", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_prefill": "0.000000", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_prefragment_max": "1048576", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_random_read_err": "0.000000", 2020-11-16T22:23:32.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_small_allocations": "0", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_default_buffered_read": "true", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_default_buffered_write": "false", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops": "0", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2020-11-16T22:23:32.457 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_max_size": "1200", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_min_size": "150", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_target_size": "500", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_freelist_blocks_per_key": "128", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2020-11-16T22:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mkfs": "true", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mount": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_mount_deep": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_umount": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_on_umount_deep": "false", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2020-11-16T22:23:32.459 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_quick_fix_threads": "2", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_gc_enable_blob_threshold": "0", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_gc_enable_total_threshold": "0", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_ignore_data_csum": "false", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_kvbackend": "rocksdb", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_collection_list_age": "60.000000", 2020-11-16T22:23:32.460 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_log_op_age": "5.000000", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_alloc_size": "0", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size": "0", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size_hdd": "65536", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_blob_size_ssd": "65536", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_defer_interval": "3.000000", 2020-11-16T22:23:32.461 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_max_deferred_txc": "32", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size": "0", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size_hdd": "4096", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_min_alloc_size_ssd": "4096", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_nid_prealloc": "1024", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size": "0", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2020-11-16T22:23:32.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_retry_disk_reads": "3", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_cf": "true", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_cfs": "m(3) O(3,0-13) L", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_coremask": "0x1", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_io_sleep": "5", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_max_io_completion": "0", 2020-11-16T22:23:32.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_spdk_mem": "512", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_sync_submit_transaction": "false", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_bytes": "67108864", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io": "0", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_throttle_trace_rate": "0.000000", 2020-11-16T22:23:32.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_tracing": "false", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_reserved": "0", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_legacy_statfs": "true", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2020-11-16T22:23:32.465 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm_path": "/usr/sbin/cephadm", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_directory_choose_policy": "random", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_mirror_action_update_interval": "2", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_cluster_require_signatures": "false", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_cluster_require_version": "1", 2020-11-16T22:23:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_require_signatures": "false", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_require_version": "1", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_service_require_signatures": "false", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_service_require_version": "1", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "cephx_sign_messages": "true", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "chdir": "", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_acl_type": "", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_asio_thread_count": "2", 2020-11-16T22:23:32.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_cache_mid": "0.750000", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_cache_size": "16384", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_caps_release_delay": "5", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_check_pool_perm": "true", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_force_sync_read": "false", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_getattr_caps": "false", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_debug_inject_tick_delay": "0", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_die_on_failed_dentry_invalidate": "true", 2020-11-16T22:23:32.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_die_on_failed_remount": "false", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_dirsize_rbytes": "true", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_force_lazyio": "false", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_fs": "", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_inject_fixed_oldest_tid": "false", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_inject_release_failure": "false", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_max_inline_size": "4096", 2020-11-16T22:23:32.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mds_namespace": "", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_metadata": "", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_gid": "-1", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_timeout": "300.000000", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mount_uid": "-1", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_mountpoint": "/", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_notify_timeout": "10", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc": "true", 2020-11-16T22:23:32.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_dirty": "104857600", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_dirty_age": "5.000000", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_max_objects": "1000", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_size": "209715200", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_oc_target_dirty": "8388608", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_permissions": "true", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_quota_df": "true", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_max_bytes": "0", 2020-11-16T22:23:32.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_max_periods": "4", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_readahead_min": "131072", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_reconnect_stale": "false", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_shutdown_timeout": "30", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_snapdir": ".snap", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_tick_interval": "1", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_trace": "", 2020-11-16T22:23:32.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_try_dentry_invalidate": "false", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_use_faked_inos": "false", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_use_random_mds": "false", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog": "false", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog_host": "127.0.0.1", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_graylog_port": "12201", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_monitors": "default=true", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog": "false", 2020-11-16T22:23:32.473 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "clog_to_syslog_level": "info", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "-", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_network": "", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_network_interface": "", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_isal": "false", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_level": "5", 2020-11-16T22:23:32.474 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zlib_winsize": "-15", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "compressor_zstd_level": "1", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crash_dir": "/var/lib/ceph/crash", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_osd_obc_lru_size": "10", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crimson_osd_scheduler_concurrency": "0", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location": "", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location_hook": "", 2020-11-16T22:23:32.475 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_location_hook_timeout": "10", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "daemonize": "false", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_allow_any_pool_priority": "false", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asok": "1/5", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asok_assert_abort": "false", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_asserts_on_shutdown": "true", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_auth": "20/20", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bdev": "1/3", 2020-11-16T22:23:32.476 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bluefs": "1/5", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_bluestore": "1/5", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_buffer": "0/1", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_cephfs_mirror": "20/20", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_civetweb": "1/10", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_client": "0/5", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_compressor": "1/5", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_context": "0/1", 2020-11-16T22:23:32.477 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_crush": "1/1", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_crypto": "1/5", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_deliberately_leak_memory": "false", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_disable_randomized_ping": "false", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_dpdk": "1/5", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_eventtrace": "1/5", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_filer": "0/1", 2020-11-16T22:23:32.478 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_filestore": "1/3", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_finisher": "1/1", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_fuse": "1/5", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_heartbeat_testing_span": "0", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_heartbeatmap": "1/5", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_immutable_obj_cache": "0/5", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_javaclient": "1/5", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_journal": "1/3", 2020-11-16T22:23:32.479 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_journaler": "0/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_kstore": "1/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_leveldb": "4/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_lockdep": "0/1", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds": "1/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_balancer": "1/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_locker": "1/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_log": "1/5", 2020-11-16T22:23:32.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_log_expire": "1/5", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mds_migrator": "1/5", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_memdb": "4/5", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mgr": "1/5", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mgrc": "1/5", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_mon": "20/20", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_monc": "0/10", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_ms": "1/1", 2020-11-16T22:23:32.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_none": "0/5", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objclass": "0/5", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objectcacher": "0/5", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_objecter": "0/1", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_optracker": "0/5", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_osd": "1/5", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_paxos": "20/20", 2020-11-16T22:23:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_perfcounter": "1/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_prioritycache": "1/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rados": "0/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd": "0/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_mirror": "0/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_pwl": "0/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rbd_replay": "0/5", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_refs": "0/0", 2020-11-16T22:23:32.483 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_reserver": "1/1", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw": "1/5", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rgw_sync": "1/5", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_rocksdb": "4/5", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_striper": "0/1", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_test": "0/5", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_throttle": "1/1", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_timer": "0/1", 2020-11-16T22:23:32.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_tp": "0/5", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "debug_version_for_testing": "", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "device_failure_prediction_mode": "none", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_graylog": "false", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_stderr": "true", 2020-11-16T22:23:32.485 INFO:teuthology.orchestra.run.smithi138.stdout: "err_to_syslog": "false", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event_tracing": "false", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "fake_statfs_for_testing": "0", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "fatal_signal_handlers": "true", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "filer_max_purge_ops": "10", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "filer_max_truncate_ops": "128", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_apply_finisher_threads": "1", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_blackhole": "false", 2020-11-16T22:23:32.486 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_btrfs_clone_range": "true", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_btrfs_snap": "true", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_caller_concurrency": "10", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_collect_device_partition_information": "true", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_commit_timeout": "600.000000", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_inject_read_err": "false", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_omap_check": "false", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_random_read_err": "0.000000", 2020-11-16T22:23:32.487 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_debug_verify_split": "false", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_dump_file": "", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_expected_throughput_ops": "200.000000", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fadvise": "true", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fail_eio": "true", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fd_cache_shards": "16", 2020-11-16T22:23:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fd_cache_size": "128", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fiemap": "false", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fiemap_threshold": "4096", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_fsync_flushes_journal_data": "false", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_index_retry_probability": "0.000000", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_inject_stall": "0", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_parallel": "false", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_trailing": "false", 2020-11-16T22:23:32.489 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_journal_writeahead": "false", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_kill_at": "0", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_alloc_hint_size": "1048576", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size": "0", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_other": "512", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs": "0", 2020-11-16T22:23:32.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_other": "2", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_inline_xattrs_xfs": "10", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_sync_interval": "5.000000", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size": "0", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_other": "1024", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2020-11-16T22:23:32.491 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_merge_threshold": "-10", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_min_sync_interval": "0.010000", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_odsync_write": "false", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_backend": "rocksdb", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_backend_path": "", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_omap_header_cache_size": "1024", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_ondisk_finisher_threads": "1", 2020-11-16T22:23:32.492 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_thread_suicide_timeout": "180", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_thread_timeout": "60", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_op_threads": "2", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_punch_hole": "false", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_high_threshhold": "0.900000", 2020-11-16T22:23:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_low_threshhold": "0.300000", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_bytes": "104857600", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_queue_max_ops": "50", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2020-11-16T22:23:32.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_seek_data_hole": "false", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_sloppy_crc": "false", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_sloppy_crc_block_size": "65536", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_splice": "false", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_split_multiple": "2", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_split_rand_factor": "20", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_update_to": "1000", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2020-11-16T22:23:32.495 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_enable": "true", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2020-11-16T22:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_xfs_extsize": "false", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "filestore_zfs_snap": "false", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "fio_dir": "/tmp/fio", 2020-11-16T22:23:32.497 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_allow_other": "true", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_atomic_o_trunc": "true", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_big_writes": "true", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_debug": "false", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_default_permissions": "false", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_disable_pagecache": "false", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_max_write": "0", 2020-11-16T22:23:32.498 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_multithreaded": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_require_active_mds": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_set_user_groups": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_move": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_read": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_splice_write": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_syncfs_on_mksnap": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "fuse_use_invalidate_cb": "true", 2020-11-16T22:23:32.499 INFO:teuthology.orchestra.run.smithi138.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "gss_target_name": "ceph", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_file": "", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_inject_failure": "0", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_interval": "5", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "host": "", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_max_inflight_ops": "128", 2020-11-16T22:23:32.500 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_max_size": "1073741824", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "immutable_object_cache_watermark": "0.100000", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "inject_early_sigterm": "false", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_aio": "true", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_align_min_size": "65536", 2020-11-16T22:23:32.501 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_block_align": "true", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_block_size": "4096", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_dio": "true", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_discard": "false", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_force_aio": "false", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_ignore_corruption": "false", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_max_write_bytes": "10485760", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_max_write_entries": "100", 2020-11-16T22:23:32.502 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_replay_from": "0", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_high_multiple": "0.000000", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_high_threshhold": "0.900000", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_low_threshhold": "0.600000", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_throttle_max_multiple": "0.000000", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_write_header_frequency": "0", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journal_zero_on_create": "false", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_prefetch_periods": "10", 2020-11-16T22:23:32.503 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_prezero_periods": "5", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "journaler_write_head_interval": "15", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "keyfile": "", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_backend": "rocksdb", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_default_stripe_size": "65536", 2020-11-16T22:23:32.504 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_fsck_on_mount": "false", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_fsck_on_mount_deep": "true", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_max_bytes": "67108864", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_max_ops": "512", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_nid_prealloc": "1024", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_onode_map_size": "1024", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_sync_submit_transaction": "false", 2020-11-16T22:23:32.505 INFO:teuthology.orchestra.run.smithi138.stdout: "kstore_sync_transaction": "false", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_block_size": "65536", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_bloom_size": "0", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_cache_size": "536870912", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_compact_on_mount": "false", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_compression": "false", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_log": "", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_log_to_ceph_log": "true", 2020-11-16T22:23:32.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_max_open_files": "0", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_paranoid": "false", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "leveldb_write_buffer_size": "33554432", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "librados_thread_count": "2", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "lockdep": "false", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "lockdep_force_backtrace": "false", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "log_coarse_timestamps": "true", 2020-11-16T22:23:32.507 INFO:teuthology.orchestra.run.smithi138.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_flush_on_exit": "false", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_graylog_host": "127.0.0.1", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_graylog_port": "12201", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_max_new": "1000", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_max_recent": "10000", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_stderr_prefix": "", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_stop_at_utilization": "0.970000", 2020-11-16T22:23:32.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": "true", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_graylog": "false", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_stderr": "false", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_syslog": "false", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "max_rotating_auth_attempts": "10", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_action_on_write_error": "1", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_asio_thread_count": "2", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_export_pin": "true", 2020-11-16T22:23:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_dirs": "true", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_interval": "5", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_fragment_size_max": "100000", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_idle_threshold": "0.000000", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_interval": "10", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_max": "-1", 2020-11-16T22:23:32.510 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_max_until": "-1", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_merge_size": "50", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_midchunk": "0.300000", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_min_rebalance": "0.100000", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_min_start": "0.200000", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_minchunk": "0.001000", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_mode": "0", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_need_max": "1.200000", 2020-11-16T22:23:32.511 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_need_min": "0.800000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_replicate_threshold": "8000.000000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_sample_interval": "3.000000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_bits": "3", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_rd": "25000.000000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_size": "10000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_split_wr": "10000.000000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_target_decay": "10.000000", 2020-11-16T22:23:32.512 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_beacon_grace": "15.000000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_beacon_interval": "4.000000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_memory_limit": "4294967296", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_mid": "0.700000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_release_free_interval": "10", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_reservation": "0.050000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_decay_rate": "1.000000", 2020-11-16T22:23:32.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_interval": "1", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache_trim_threshold": "65536", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_delegate_inos_pct": "50", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_prealloc_inos": "1000", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2020-11-16T22:23:32.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_damage_table_max_entries": "10000", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_auth_pins": "false", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_frag": "false", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_scatterstat": "false", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_debug_subtrees": "false", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_decay_halflife": "5.000000", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_default_dir_hash": "2", 2020-11-16T22:23:32.515 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_defer_session_stale": "true", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_deny_all_reconnect": "false", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_keys_per_op": "16384", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dir_max_commit_size": "10", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dirstat_min_interval": "1.000000", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_after_rejoin": "false", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_on_map": "false", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_threshold_file": "0", 2020-11-16T22:23:32.516 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_early_reply": "true", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_enable_op_tracker": "true", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_enforce_unique_name": "true", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_distributed": "true", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_random": "true", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_export_ephemeral_random_max": "0.010000", 2020-11-16T22:23:32.517 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_forward_all_requests_to_auth": "false", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_freeze_tree_timeout": "30.000000", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_health_cache_threshold": "1.500000", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_health_summarize_threshold": "10", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_heartbeat_grace": "15.000000", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_migrator_session_race": "false", 2020-11-16T22:23:32.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_join_fs": "", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_journal_format": "1", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_create_at": "0", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_export_at": "0", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_import_at": "0", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_at": "0", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_expire_at": "0", 2020-11-16T22:23:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_journal_replay_at": "0", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_link_at": "0", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_mdstable_at": "0", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_openc_at": "0", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_kill_rename_at": "0", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_events_per_segment": "1024", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_max_events": "-1", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_max_segments": "128", 2020-11-16T22:23:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_pause": "false", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_segment_size": "0", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_skip_corrupt_events": "false", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log_warn_factor": "2.000000", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_caps_per_client": "1048576", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_completed_flushes": "100000", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_completed_requests": "100000", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_export_size": "20971520", 2020-11-16T22:23:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_file_recover": "32", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_files": "64", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_ops": "8192", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_retries_on_remount_failure": "5", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_scrub_ops_in_progress": "5", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_snaps_per_dir": "100", 2020-11-16T22:23:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_max_xattr_pairs_size": "65536", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_metrics_update_interval": "2", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_min_caps_per_client": "100", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_min_caps_working_set": "10000", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mon_shutdown_timeout": "5.000000", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_numa_node": "-1", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_oft_prefetch_dirfrags": "true", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_complaint_time": "30.000000", 2020-11-16T22:23:32.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_history_duration": "600", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_history_size": "20", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_op_log_threshold": "5", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_ping_grace": "15", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_ping_interval": "5", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_global_max_decay_threshold": "65536", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_caps": "5000", 2020-11-16T22:23:32.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_decay_rate": "2.500000", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_max_decay_threshold": "16384", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_warning_decay_rate": "60.000000", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_recall_warning_threshold": "32768", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_reconnect_timeout": "45.000000", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_replay_interval": "1.000000", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_replay_unsafe_with_closed_session": "false", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_request_load_average_decay_rate": "60.000000", 2020-11-16T22:23:32.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_root_ino_gid": "0", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_root_ino_uid": "0", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_scatter_nudge_interval": "5.000000", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_blocklist_on_evict": "true", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_blocklist_on_timeout": "true", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cache_liveness_magnitude": "10", 2020-11-16T22:23:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_cap_acquisition_throttle": "500000", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessionmap_keys_per_op": "1024", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_shutdown_check": "0", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_skip_ino": "0", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_max_uid": "4294967294", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_min_uid": "0", 2020-11-16T22:23:32.527 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_snap_rstat": "false", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_task_status_update_interval": "2.000000", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_thrash_exports": "0", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_thrash_fragments": "0", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_tick_interval": "5.000000", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_verify_backtrace": "1", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_verify_scatter": "false", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_wipe_ino_prealloc": "false", 2020-11-16T22:23:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_wipe_sessions": "false", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool_debug": "false", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_debug_omit_block_device_write": "false", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_device_bytes": "1073741824", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_page_set": "false", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "memstore_page_size": "65536", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_bytes": "134217728", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_messages": "512", 2020-11-16T22:23:32.529 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_connect_retry_interval": "1.000000", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_disabled_modules": "", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_initial_modules": "restful iostat", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mds_bytes": "134217728", 2020-11-16T22:23:32.530 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mds_messages": "128", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mon_bytes": "134217728", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_mon_messages": "128", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_osd_bytes": "536870912", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_osd_messages": "8192", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_service_beacon_grace": "60.000000", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_stats_period": "5", 2020-11-16T22:23:32.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_stats_threshold": "5", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_tick_period": "2", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_accept_timeout_factor": "2.000000", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_allow_pool_delete": "true", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_allow_pool_size_one": "true", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_auth_validate_all_caps": "true", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2020-11-16T22:23:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_bytes": "104857600", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_directed_command_retry": "2", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval": "3.000000", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_hunt_parallel": "3", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_log_interval": "1.000000", 2020-11-16T22:23:32.533 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_max_log_entries_per_message": "1000", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_ping_interval": "10.000000", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_client_ping_timeout": "30.000000", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clock_drift_allowed": "1.000000", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_file_level": "debug", 2020-11-16T22:23:32.534 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_file": "true", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog": "false", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_graylog_port": "12201", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_stderr": "false", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog": "default=false", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cluster_log_to_syslog_level": "info", 2020-11-16T22:23:32.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_bootstrap": "false", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_start": "false", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_compact_on_trim": "true", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_con_tracker_persist_interval": "10", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_con_tracker_score_halflife": "43200", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_config_key_max_entry_size": "65536", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_cpu_threads": "4", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_crush_min_required_version": "hammer", 2020-11-16T22:23:32.536 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_daemon_bytes": "419430400", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_avail_crit": "5", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_avail_warn": "5", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_data_size_warn": "16106127360", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_block_osdmap_trim": "false", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_deprecated_as_obsolete": "false", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_json": "false", 2020-11-16T22:23:32.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_dump_transactions": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_extra_checks": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_initial_persistent_features": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_octopus": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_no_require_pacific": "false", 2020-11-16T22:23:32.538 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_delta_reset_interval": "10.000000", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_dns_srv_name": "ceph-mon", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_election_default_strategy": "1", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_election_timeout": "5.000000", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ping_divisor": "2", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_elector_ping_timeout": "2.000000", 2020-11-16T22:23:32.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_enable_op_tracker": "true", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_fake_pool_delete": "false", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_force_quorum_join": "false", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_globalid_prealloc": "10000", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_detail_to_clog": "false", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_log_update_period": "5", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_max_detail": "50", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog": "true", 2020-11-16T22:23:32.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog_interval": "600", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_host": "172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790],[v2:172.21.15.138:3302,v1:172.21.15.138:6791]", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_host_override": "", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_initial_members": "", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2020-11-16T22:23:32.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_transaction_delay_max": "10.000000", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_keyvaluedb": "rocksdb", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease": "5.000000", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_lease_renew_interval_factor": "0.600000", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_log_max_summary": "50", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_log_entries_per_event": "4096", 2020-11-16T22:23:32.542 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_log_epochs": "500", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_mdsmap_epochs": "500", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_mgrmap_epochs": "500", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_osd": "10000", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_pg_per_osd": "10000", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_pool_pg_num": "65536", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_max_snap_prune_per_epoch": "100", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_blocklist_interval": "86400.000000", 2020-11-16T22:23:32.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_force_trim_to": "0", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mds_skip_sanity": "false", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_memory_autotune": "true", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_memory_target": "2147483648", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_beacon_grace": "30", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_digest_period": "5", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_inactive_grace": "60", 2020-11-16T22:23:32.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_mkfs_grace": "240", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_min_osdmap_epochs": "500", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_complaint_time": "30", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_duration": "600", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_size": "20", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_slow_op_size": "20", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_history_slow_op_threshold": "10", 2020-11-16T22:23:32.545 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_op_log_threshold": "5", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_adjust_down_out_interval": "true", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_in": "false", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_auto_mark_new_in": "true", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2020-11-16T22:23:32.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_cache_size": "500", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_cache_size_min": "134217728", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_crush_smoke_test": "true", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_destroyed_out_interval": "600", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_down_out_interval": "600", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_down_out_subtree_limit": "rack", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2020-11-16T22:23:32.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_force_trim_to": "0", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_full_ratio": "0.950000", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_halflife": "3600", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_max_interval": "300", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_laggy_weight": "0.300000", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_max_creating_pgs": "1024", 2020-11-16T22:23:32.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_max_initial_pgs": "1024", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_down_reporters": "2", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_in_ratio": "0.750000", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_min_up_ratio": "0.300000", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_nearfull_ratio": "0.850000", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp": "true", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2020-11-16T22:23:32.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_report_timeout": "900", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_reporter_subtree_level": "osd", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_warn_num_repaired": "10", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osd_warn_op_age": "32.000000", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_enabled": "true", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_interval": "10", 2020-11-16T22:23:32.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_min": "10000", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_osdmap_full_prune_txsize": "100", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_stuck_threshold": "60", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_objects": "10000", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_per_osd": "0", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pg_warn_min_pool_objects": "1000", 2020-11-16T22:23:32.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pool_quota_crit_threshold": "0", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_pool_quota_warn_threshold": "0", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_probe_timeout": "2.000000", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_max_change": "0.050000", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_max_osds": "4", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_min_bytes_per_osd": "10", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_reweight_min_pgs_per_osd": "4", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2020-11-16T22:23:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_interval": "86400", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_max_keys": "100", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_scrub_timeout": "300", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_session_timeout": "300", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_smart_report_timeout": "5", 2020-11-16T22:23:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stat_smooth_intervals": "6", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_pool_min_size": "2", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_pool_size": "4", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_subscribe_interval": "86400.000000", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_debug": "false", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_max_payload_keys": "2000", 2020-11-16T22:23:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_max_payload_size": "1048576", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_provider_kill_at": "0", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_requester_kill_at": "0", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_sync_timeout": "60.000000", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_target_pg_per_osd": "100", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_tick_interval": "5", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_timecheck_interval": "300.000000", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_timecheck_skew_interval": "30.000000", 2020-11-16T22:23:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_older_version_delay": "604800.000000", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_misplaced": "false", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_older_version": "true", 2020-11-16T22:23:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_no_app": "true", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_no_redundancy": "false", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_on_too_few_osds": "false", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2020-11-16T22:23:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": "", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_max_op_threads": "5", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_op_threads": "3", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_buffer_size": "131072", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_cm": "false", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_device_name": "", 2020-11-16T22:23:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_dscp": "96", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_enable_hugepage": "false", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_gid_idx": "0", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_local_gid": "", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_polling_us": "1000", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_port_num": "1", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_receive_buffers": "32768", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_receive_queue_len": "4096", 2020-11-16T22:23:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_roce_ver": "1", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_send_buffers": "1024", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_sl": "3", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_support_srq": "true", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_async_rdma_type": "ib", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_before_connect": "false", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_ipv4": "true", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_ipv6": "false", 2020-11-16T22:23:32.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_msgr1": "true", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_msgr2": "true", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_port_max": "7300", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_port_min": "6800", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_prefer_ipv4": "false", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_retry_count": "3", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_bind_retry_delay": "5", 2020-11-16T22:23:32.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_client": "false", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mds": "false", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mgr": "false", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_mon": "false", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_blackhole_osd": "false", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_client_mode": "crc secure", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_cluster_mode": "crc secure", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_cluster_type": "", 2020-11-16T22:23:32.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_connection_idle_timeout": "900", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_connection_ready_timeout": "10", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_crc_data": "true", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_crc_header": "true", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_bad_msg": "false", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_bug": "true", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_old_message": "true", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_skipped_message": "false", 2020-11-16T22:23:32.563 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_die_on_unhandled_msg": "false", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dispatch_throttle_bytes": "104857600", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_coremask": "0xF", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_debug_allow_loopback": "false", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_host_ipv4_addr": "", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hugepages": "", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hw_flow_control": "true", 2020-11-16T22:23:32.564 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_lro": "true", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_memory_channel": "4", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_pmd": "", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_port_id": "0", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2020-11-16T22:23:32.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dump_corrupt_message_level": "1", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_dump_on_send": "false", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_initial_backoff": "0.200000", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_max": "1.000000", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_msg_type": "", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_probability": "0.000000", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_delay_type": "", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_internal_delays": "0.000000", 2020-11-16T22:23:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_inject_socket_failures": "0", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_learn_addr_from_peer": "true", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_max_accept_failures": "4", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_max_backoff": "15.000000", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_client_mode": "secure crc", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_cluster_mode": "secure crc", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_mon_service_mode": "secure crc", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2020-11-16T22:23:32.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_pq_min_cost": "65536", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_public_type": "", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_service_mode": "crc secure", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_listen_backlog": "512", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_nodelay": "true", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_prefetch_max_size": "4096", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_tcp_rcvbuf": "0", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ms_type": "async+posix", 2020-11-16T22:23:32.568 INFO:teuthology.orchestra.run.smithi138.stdout: "no_config_file": "false", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_completion_locks_per_session": "32", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_debug_inject_relock_delay": "false", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inflight_op_bytes": "104857600", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inflight_ops": "1024", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_inject_no_watch_ping": "false", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_retry_writes_after_first_reply": "false", 2020-11-16T22:23:32.569 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_tick_interval": "5.000000", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter_timeout": "10.000000", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "objectstore_blackhole": "false", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "openssl_engine_opts": "", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_delay_time": "5.000000", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_hist_halflife": "1000", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_max_low_ops": "2", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_max_ops": "4", 2020-11-16T22:23:32.570 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_min_evict_effort": "0.100000", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_quantize_effort": "0.100000", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_agent_slop": "0.020000", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_allow_recovery_below_min_size": "true", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_asio_thread_count": "2", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_async_recovery_min_cost": "100", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_auto_mark_unfound_lost": "false", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_retry_interval": "30.000000", 2020-11-16T22:23:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_scan_max": "512", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backfill_scan_min": "64", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_degraded": "false", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_peering": "false", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_backoff_on_unfound": "true", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_beacon_report_interval": "300", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_duration": "30", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_large_size_max_throughput": "104857600", 2020-11-16T22:23:32.572 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_max_block_size": "67108864", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bench_small_size_max_iops": "100", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_blkin_trace_all": "false", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_check_for_log_corruption": "false", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_check_max_object_name_len_on_startup": "true", 2020-11-16T22:23:32.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_class_update_on_start": "true", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_message_cap": "0", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_message_size_cap": "524288000", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_op_priority": "63", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_client_watch_timeout": "30", 2020-11-16T22:23:32.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_command_max_records": "256", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_command_thread_suicide_timeout": "900", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_command_thread_timeout": "600", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_compact_on_start": "false", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_copyfrom_max_chunk": "8388608", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_chooseleaf_type": "0", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_initial_weight": "-1.000000", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_update_on_start": "true", 2020-11-16T22:23:32.575 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_crush_update_weight_set": "true", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_drop_ping_duration": "0", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_drop_ping_probability": "0.000000", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_feed_pullee": "-1", 2020-11-16T22:23:32.576 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_copyfrom_error": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_misdirected_ops": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_no_acting_change": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_no_purge_strays": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_op_order": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_pg_log_writeout": "false", 2020-11-16T22:23:32.577 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_pretend_recovery_active": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_random_push_read_error": "0.000000", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_shutdown": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_cached_snaps": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_missing_on_start": "false", 2020-11-16T22:23:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_snaps": "false", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_debug_verify_stray_on_activate": "false", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_interval": "604800.000000", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_keys": "1024", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_stride": "524288", 2020-11-16T22:23:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_default_data_pool_replay_window": "5", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_default_notify_timeout": "30", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep": "0.000000", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_hdd": "5.000000", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_hybrid": "2.000000", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_delete_sleep_ssd": "0.000000", 2020-11-16T22:23:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_discard_disconnected_ops": "true", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_enable_op_tracker": "true", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_failsafe_full_ratio": "0.970000", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_fail_on_connection_refused": "true", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_info": "true", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_fast_shutdown": "true", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_find_best_info_ignore_history_les": "false", 2020-11-16T22:23:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_force_auth_primary_missing_objects": "100", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_function_tracing": "false", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_grace": "20", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_interval": "6", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_peers": "10", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_min_size": "2000", 2020-11-16T22:23:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_stale": "600", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_max_size": "100000", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_min_size": "1000", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_hit_set_namespace": ".ceph-internal", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_ignore_stale_divergent_priors": "false", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_inject_failure_on_pg_removal": "false", 2020-11-16T22:23:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal_flush_on_shutdown": "true", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_journal_size": "5120", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_kill_backfill_at": "0", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_loop_before_reset_tphandle": "64", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_cache_size": "50", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_dedup": "true", 2020-11-16T22:23:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_message_max": "40", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_message_max_bytes": "10485760", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_map_share_max_epochs": "40", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_attr_name_len": "100", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_attr_size": "0", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_backfills": "1", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_markdown_count": "5", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_markdown_period": "600", 2020-11-16T22:23:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_name_len": "2048", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_namespace_len": "256", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_object_size": "134217728", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_omap_entries_per_request": "1024", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_blocked_by": "16", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_log_entries": "10000", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2020-11-16T22:23:32.586 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_pgls": "1024", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_push_cost": "8388608", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_push_objects": "10", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_scrubs": "1", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_trimming_pgs": "2", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_write_op_reply_len": "32", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_max_write_size": "90", 2020-11-16T22:23:32.587 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_lim": "999999", 2020-11-16T22:23:32.588 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_res": "1", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mclock_scheduler_client_wgt": "1", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_base": "805306368", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_cache_min": "134217728", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_cache_resize_interval": "1.000000", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_expected_fragmentation": "0.150000", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_target": "4294967296", 2020-11-16T22:23:32.589 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_min_pg_log_entries": "250", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_min_recovery_priority": "0", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_ack_timeout": "30.000000", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_heartbeat_interval": "30", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_report_interval": "5", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_report_max_in_flight": "2", 2020-11-16T22:23:32.590 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mon_shutdown_timeout": "5.000000", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_num_cache_shards": "32", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_num_op_tracker_shard": "32", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_auto_affinity": "true", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_node": "-1", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_numa_prefer_iface": "true", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_object_clean_region_max_num_intervals": "10", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objecter_finishers": "1", 2020-11-16T22:23:32.591 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore": "bluestore", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore_fuse": "false", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_objectstore_tracing": "false", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_complaint_time": "30.000000", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_duration": "600", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_size": "20", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_slow_op_size": "20", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2020-11-16T22:23:32.592 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_log_threshold": "5", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards": "0", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards_hdd": "5", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_shards_ssd": "8", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard": "0", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2020-11-16T22:23:32.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_pq_min_cost": "65536", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_queue": "wpq", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_queue_cut_off": "high", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_thread_suicide_timeout": "150", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_op_thread_timeout": "15", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_open_classes_on_start": "true", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_os_flags": "0", 2020-11-16T22:23:32.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_peering_op_priority": "255", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_delete_cost": "1048576", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_delete_priority": "5", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_dups_tracked": "3000", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_trim_max": "10000", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_log_trim_min": "100", 2020-11-16T22:23:32.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_object_context_cache_count": "64", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pg_stat_report_interval_max": "500", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_min_evict_age": "0", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_min_flush_age": "0", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2020-11-16T22:23:32.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_crush_rule": "-1", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_ec_fast_read": "false", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_hashpspool": "true", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nodelete": "false", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nopgchange": "false", 2020-11-16T22:23:32.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flag_nosizechange": "false", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_flags": "0", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_min_size": "0", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pg_num": "32", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_pgp_num": "0", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2020-11-16T22:23:32.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_size": "2", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_default_type": "replicated", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pool_use_gmt_hitset": "true", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_push_per_object_cost": "1000", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_read_ec_check_for_errors": "false", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recover_clone_overlap": "true", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recover_clone_overlap_limit": "10", 2020-11-16T22:23:32.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_cost": "20971520", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_delay_start": "0.000000", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active": "0", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active_hdd": "3", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_active_ssd": "10", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_chunk": "8388608", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2020-11-16T22:23:32.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_max_single_start": "1", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_op_priority": "3", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_op_warn_multiple": "16", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_priority": "5", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_retry_interval": "30.000000", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep": "0.000000", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_hdd": "0.100000", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2020-11-16T22:23:32.601 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_recovery_sleep_ssd": "0.000000", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_repair_during_recovery": "false", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_requested_scrub_priority": "120", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_rollback_to_cluster_snap": "", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_auto_repair": "false", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_auto_repair_num_errors": "5", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_backoff_ratio": "0.660000", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_begin_hour": "0", 2020-11-16T22:23:32.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_begin_week_day": "0", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_chunk_max": "25", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_chunk_min": "5", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_cost": "52428800", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_during_recovery": "false", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_end_hour": "0", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_end_week_day": "0", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_extended_sleep": "0.000000", 2020-11-16T22:23:32.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_invalid_stats": "true", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_load_threshold": "0.500000", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_max_interval": "604800.000000", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_max_preemptions": "5", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_min_interval": "86400.000000", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_priority": "5", 2020-11-16T22:23:32.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_scrub_sleep": "0.000000", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_shutdown_pgref_assert": "false", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_skip_data_digest": "false", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_smart_report_timeout": "5", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_cost": "1048576", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_priority": "5", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep": "0.000000", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2020-11-16T22:23:32.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_stats_ack_timeout_decay": "0.900000", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_stats_ack_timeout_factor": "2.000000", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_target_transaction_size": "30", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_count": "4", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2020-11-16T22:23:32.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_default_cache_mode": "writeback", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tier_promote_max_objects_sec": "25", 2020-11-16T22:23:32.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_tracing": "false", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_use_stale_snap": "false", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osdc_blkin_trace_all": "false", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_kill_at": "0", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_max_join_drift": "10", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_min": "500", 2020-11-16T22:23:32.608 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_min_wait": "0.050000", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_propose_interval": "1.000000", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_service_trim_max": "500", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_service_trim_min": "250", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_stash_full_interval": "25", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_trim_max": "500", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "paxos_trim_min": "250", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "perf": "true", 2020-11-16T22:23:32.609 INFO:teuthology.orchestra.run.smithi138.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin_crypto_accelerator": "crypto_isal", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin_dir": "/usr/lib64/ceph", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "-", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrv": "", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "public_bind_addr": "-", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "public_network": "", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "public_network_interface": "", 2020-11-16T22:23:32.610 INFO:teuthology.orchestra.run.smithi138.stdout: "qat_compressor_enabled": "false", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_mon_op_timeout": "0", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_osd_op_timeout": "0", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rados_tracing": "false", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_atime_update_interval": "60", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_balance_parent_reads": "false", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_balance_snap_reads": "false", 2020-11-16T22:23:32.611 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blkin_trace_all": "false", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blocklist_expire_seconds": "0", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_blocklist_on_break_lock": "true", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache": "true", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_block_writes_upfront": "false", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty": "25165824", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty_age": "1.000000", 2020-11-16T22:23:32.612 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_max_dirty_object": "0", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_policy": "writearound", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_size": "33554432", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_target_dirty": "16777216", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_cache_writethrough_until_flush": "true", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_clone_copy_on_read": "false", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_compression_hint": "none", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_concurrent_management_ops": "10", 2020-11-16T22:23:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_config_pool_override_update_timestamp": "0", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_clone_format": "auto", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_data_pool": "", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_features": "61", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_format": "2", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_map_options": "", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_order": "22", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_pool": "rbd", 2020-11-16T22:23:32.614 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_stripe_count": "0", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_default_stripe_unit": "0", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_disable_zero_copy_writes": "true", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_discard_granularity_bytes": "65536", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_discard_on_zeroed_write_same": "true", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_enable_alloc_hint": "true", 2020-11-16T22:23:32.615 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_io_scheduler": "simple", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_commit_age": "5.000000", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_max_payload_bytes": "16384", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_age": "0.000000", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_bytes": "1048576", 2020-11-16T22:23:32.616 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_flush_interval": "0", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_order": "24", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_pool": "", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_journal_splay_width": "4", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_localize_parent_reads": "false", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_localize_snap_reads": "false", 2020-11-16T22:23:32.617 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_type": "simple", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2020-11-16T22:23:32.618 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_image_state_check_interval": "30", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_autotune": "true", 2020-11-16T22:23:32.619 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_base": "805306368", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_min": "134217728", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_memory_target": "4294967296", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_perf_stats_prio": "5", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2020-11-16T22:23:32.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_delete_delay": "0", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_replay_delay": "0", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_to_trash_on_remove": "false", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2020-11-16T22:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_mtime_update_interval": "60", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_non_blocking_aio": "true", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_op_thread_timeout": "60", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_op_threads": "1", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_parent_cache_enabled": "false", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_plugins": "", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_burst": "0", 2020-11-16T22:23:32.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_burst_seconds": "1", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_bps_limit": "0", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_burst": "0", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_burst_seconds": "1", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_iops_limit": "0", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_burst": "0", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_bps_limit": "0", 2020-11-16T22:23:32.623 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_burst": "0", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_read_iops_limit": "0", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_schedule_tick_min": "50", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_burst": "0", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_bps_limit": "0", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_burst": "0", 2020-11-16T22:23:32.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_qos_write_iops_limit": "0", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_quiesce_notification_attempts": "10", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_read_from_replica_policy": "default", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_max_bytes": "524288", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_readahead_trigger_requests": "10", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_request_timed_out_seconds": "30", 2020-11-16T22:23:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_rwl_enabled": "false", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_rwl_log_periodic_stats": "false", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_rwl_path": "/tmp", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_rwl_size": "1073741824", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_skip_partial_discard": "true", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_tracing": "false", 2020-11-16T22:23:32.626 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_validate_names": "true", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_validate_pool": "true", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "restapi_base_url": "", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "restapi_log_level": "", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_acl_grants_max_num": "100", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_admin_entry": "admin", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_barbican_url": "", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2020-11-16T22:23:32.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_default_quota_max_size": "-1", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_index_max_aio": "128", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_quota_cache_size": "10000", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_bucket_quota_ttl": "600", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_enabled": "true", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_expiry_interval": "900", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cache_lru_size": "10000", 2020-11-16T22:23:32.628 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_content_length_compat": "false", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_copy_obj_progress": "true", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cors_rules_max_num": "100", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_cross_domain_policy": "", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_default_encryption_key": "", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_require_ssl": "true", 2020-11-16T22:23:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_suppress_logs": "true", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_addr": "", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_auth": "token", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_namespace": "", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_prefix": "", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_secret_engine": "transit", 2020-11-16T22:23:32.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_crypt_vault_token_file": "", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_low_speed_limit": "1024", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_low_speed_time": "300", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_curl_wait_timeout_ms": "1000", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_backing": "auto", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_changes_size": "1000", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_num_shards": "128", 2020-11-16T22:23:32.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_obj_prefix": "data_log", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_log_window": "30", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_data_notify_interval_msec": "200", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_realm_info_oid": "default.realm", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_region_info_oid": "default.region", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_zone_info_oid": "default.zone", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_defer_to_bucket_acls": "", 2020-11-16T22:23:32.632 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_delete_multi_obj_max_num": "1000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_lim": "0.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_res": "100.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_lim": "0.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_res": "200.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2020-11-16T22:23:32.633 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_lim": "0.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_res": "500.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_data_wgt": "500.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_res": "500.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dns_name": "", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dns_s3website_name": "", 2020-11-16T22:23:32.634 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_dynamic_resharding": "true", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_gc_threads": "true", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_lc_threads": "true", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_ops_log": "false", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_quota_threads": "true", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_static_website": "false", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enable_usage_log": "false", 2020-11-16T22:23:32.635 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_enforce_swift_acls": "true", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_exit_timeout_secs": "120", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_expose_bucket": "false", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_extended_http_attrs": "", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_fcgi_socket_backlog": "1024", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_frontends": "beast port=7480", 2020-11-16T22:23:32.636 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_concurrent_io": "10", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_deferred": "50", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_objs": "32", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_queue_size": "134213632", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_max_trim_chunk": "16", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_obj_min_wait": "7200", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_processor_max_time": "3600", 2020-11-16T22:23:32.637 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_gc_processor_period": "3600", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_get_obj_max_req_size": "4194304", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_get_obj_window_size": "16777216", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_healthcheck_disabling_path": "", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_host": "", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ignore_get_invalid_range": "false", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_init_timeout": "300", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2020-11-16T22:23:32.638 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_accepted_admin_roles": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_domain": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_password": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_password_path": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_project": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_tenant": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_token": "", 2020-11-16T22:23:32.639 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_token_path": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_admin_user": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_api_version": "2", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_domain": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_password": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_project": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_tenant": "", 2020-11-16T22:23:32.640 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_barbican_user": "", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_implicit_tenants": "false", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_token_cache_size": "10000", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_url": "", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_keystone_verify_ssl": "true", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_debug_interval": "-1", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_lock_max_time": "90", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_objs": "32", 2020-11-16T22:23:32.641 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_rules": "1000", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_worker": "3", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_max_wp_worker": "3", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lc_thread_delay": "0", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_dnattr": "uid", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_searchfilter": "", 2020-11-16T22:23:32.642 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ldap_uri": "ldaps://", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_list_bucket_min_readahead": "1000", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_list_buckets_max_chunk": "1000", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_http_headers": "", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_nonexistent_bucket": "false", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2020-11-16T22:23:32.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_log_object_name_utc": "false", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attr_name_len": "0", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attr_size": "0", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_attrs_num_in_req": "0", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_chunk_size": "4194304", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_concurrent_requests": "1024", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_dynamic_shards": "1999", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_listing_results": "1000", 2020-11-16T22:23:32.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_notify_retries": "3", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_objs_per_shard": "100000", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_put_param_size": "1048576", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_put_size": "5368709120", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_max_slo_entries": "1000", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_md_log_max_shards": "64", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_md_notify_interval_msec": "200", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_mime_types_file": "/etc/mime.types", 2020-11-16T22:23:32.645 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_mp_lock_max_time": "600", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_multipart_min_part_size": "5242880", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_multipart_part_upload_limit": "10000", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_fhcache_partitions": "3", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_fhcache_size": "2017", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_lru_lanes": "5", 2020-11-16T22:23:32.646 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_max_gc": "300", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_namespace_expire_secs": "300", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_gc_threads": "false", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_lc_threads": "false", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_quota_threads": "false", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_run_sync_thread": "false", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_s3_fast_attrs": "false", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_nfs_write_completion_interval_s": "10", 2020-11-16T22:23:32.647 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_num_async_rados_threads": "32", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_num_control_oids": "8", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_num_rados_handles": "1", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_numa_node": "-1", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_obj_stripe_size": "4194304", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_obj_tombstone_cache_size": "1000", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_chunk_size": "100", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_gc_interval": "600", 2020-11-16T22:23:32.648 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_objexp_hints_num_shards": "127", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_olh_pending_timeout_sec": "3600", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_thread_suicide_timeout": "0", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_thread_timeout": "600", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_op_tracing": "false", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_token": "", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_url": "", 2020-11-16T22:23:32.649 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_opa_verify_ssl": "true", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_data_backlog": "5242880", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_rados": "true", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_ops_log_socket_path": "", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_override_bucket_index_max_shards": "0", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_push_interval": "2.000000", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_push_interval_max": "30.000000", 2020-11-16T22:23:32.650 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_period_root_pool": ".rgw.root", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_port": "", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_print_continue": "true", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_print_prohibited_content_length": "false", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_put_obj_max_window_size": "67108864", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_put_obj_min_window_size": "16777216", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_pool_pg_num_min": "8", 2020-11-16T22:23:32.651 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_pool_recovery_priority": "5", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rados_tracing": "false", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_realm": "", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_realm_root_pool": ".rgw.root", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_region": "", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_region_root_pool": ".rgw.root", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_relaxed_region_enforcement": "false", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_relaxed_s3_bucket_names": "false", 2020-11-16T22:23:32.652 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_request_uri": "", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_batch_size": "64", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_bucket_lock_duration": "360", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_max_aio": "128", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_num_logs": "16", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_reshard_thread_interval": "600", 2020-11-16T22:23:32.653 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_resolve_cname": "false", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_rest_getusage_op_compat": "false", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_run_sync_thread": "true", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_order": "sts, external, local", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_keystone": "false", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_ldap": "false", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_rados": "true", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_auth_use_sts": "false", 2020-11-16T22:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_s3_success_create_obj_status": "0", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_safe_max_objects_per_shard": "102400", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_scheduler_type": "throttler", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_script_uri": "", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_service_provider_name": "", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_shard_warning_threshold": "90.000000", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_socket_path": "", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_client_id": "", 2020-11-16T22:23:32.655 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_client_secret": "", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_entry": "sts", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_key": "sts", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_max_session_duration": "43200", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_min_session_duration": "900", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sts_token_introspection_url": "", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_account_in_url": "false", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_auth_entry": "auth", 2020-11-16T22:23:32.656 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_auth_url": "", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_custom_header": "", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_enforce_content_length": "false", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_need_stats": "true", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_tenant_name": "", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_token_expiration": "86400", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_url": "", 2020-11-16T22:23:32.657 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_url_prefix": "swift", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_swift_versioning_enabled": "false", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_lease_period": "120", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_interval": "1200", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_max_buckets": "16", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2020-11-16T22:23:32.658 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_obj_etag_verify": "false", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_history_size": "4096", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_per_node_log_size": "32", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_thread_pool_size": "512", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_comment": "", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_createby": "", 2020-11-16T22:23:32.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_encoding": "", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_flag": "false", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_origin": "", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_sha_unit": "524288", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_torrent_tracker": "", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_trust_forwarded_https": "false", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_log_flush_threshold": "1024", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_log_tick_interval": "30", 2020-11-16T22:23:32.660 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_max_shards": "32", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_usage_max_user_shards": "1", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_use_opa_authz": "false", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_default_quota_max_objects": "-1", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_default_quota_max_size": "-1", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_max_buckets": "1000", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_idle_users": "false", 2020-11-16T22:23:32.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_interval": "86400", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_quota_sync_wait_time": "86400", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_user_unique_email": "true", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_verify_ssl": "true", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_website_routing_rules_max_num": "50", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zone": "", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zone_root_pool": ".rgw.root", 2020-11-16T22:23:32.662 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zonegroup": "", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_block_size": "4096", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_bloom_bits_per_key": "20", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_row_ratio": "0.000000", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_shard_bits": "4", 2020-11-16T22:23:32.663 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_size": "536870912", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_cache_type": "binned_lru", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_compaction_stats": "false", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_extended_stats": "false", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_collect_memory_stats": "false", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_delete_range_threshold": "1048576", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_index_type": "binary_search", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_log_to_ceph_log": "true", 2020-11-16T22:23:32.664 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_metadata_block_size": "4096", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_partition_filters": "false", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_perf": "false", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "rotating_keys_bootstrap_timeout": "30", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "rotating_keys_renewal_timeout": "10", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "run_dir": "/var/run/ceph", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "setgroup": "", 2020-11-16T22:23:32.665 INFO:teuthology.orchestra.run.smithi138.stdout: "setuser": "", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "setuser_match_path": "", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_misplaced_ratio": "0.050000", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "thp": "false", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "threadpool_default_timeout": "60", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "threadpool_empty_queue_max_wait": "2", 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout: "throttler_perf_counter": "true" 2020-11-16T22:23:32.666 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:23:32.668 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2020-11-16T22:23:32.798 INFO:teuthology.orchestra.run.smithi138.stdout:mon.a: {} 2020-11-16T22:23:32.808 INFO:teuthology.orchestra.run.smithi138.stdout:mon.b: {} 2020-11-16T22:23:32.818 INFO:teuthology.orchestra.run.smithi138.stdout:mon.c: {} 2020-11-16T22:23:32.819 INFO:teuthology.orchestra.run.smithi138.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T22:23:32.819 INFO:teuthology.orchestra.run.smithi138.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T22:23:32.819 INFO:teuthology.orchestra.run.smithi138.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T22:23:32.829 INFO:tasks.thrashosds.thrasher:Unable to test ceph-objectstore-tool, powercycle testing 2020-11-16T22:23:32.829 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-11-16T22:23:32.851 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-11-16T22:23:32.851 INFO:tasks.ceph_fuse:config is None 2020-11-16T22:23:32.851 INFO:tasks.ceph_fuse:new config is {'client.0': None} 2020-11-16T22:23:32.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:32.852 INFO:teuthology.orchestra.run.smithi138:> ip netns list 2020-11-16T22:23:32.853 INFO:tasks.thrashosds.thrasher:starting do_thrash 2020-11-16T22:23:32.853 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:23:32.853 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:23:32.853 INFO:tasks.thrashosds.thrasher:inject_pause on 2 2020-11-16T22:23:32.853 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:23:32.854 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:23:32.854 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2020-11-16T22:23:32.854 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2020-11-16T22:23:32.855 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2020-11-16T22:23:32.855 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:32.855 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2020-11-16T22:23:32.856 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:32.856 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2020-11-16T22:23:32.856 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:23:32.858 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:32.869+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:32.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:32.872 INFO:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2020-11-16T22:23:32.942 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2020-11-16T22:23:32.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:23:32.944 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-11-16T22:23:32.944 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:23:32.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2020-11-16T22:23:32.945 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:23:32.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:23:32.945 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:23:32.945 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-16T22:23:32.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:23:32.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:23:32.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:32.946 INFO:teuthology.orchestra.run.smithi138:> ip addr 2020-11-16T22:23:32.956 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:32.969+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:32.995 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:32.996 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-11-16T22:23:32.996 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:33.002 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:23:33.002 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:23:33.002 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:23:33.002 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:23:33.002 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-16T22:23:33.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42686sec preferred_lft 42686sec 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2020-11-16T22:23:33.004 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:23:33.005 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-16T22:23:33.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.005 INFO:teuthology.orchestra.run.smithi138:> (cd / && exec sudo bash -c ' 2020-11-16T22:23:33.005 INFO:teuthology.orchestra.run.smithi138:> set -e 2020-11-16T22:23:33.006 INFO:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2020-11-16T22:23:33.006 INFO:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2020-11-16T22:23:33.006 INFO:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2020-11-16T22:23:33.006 INFO:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-16T22:23:33.007 INFO:teuthology.orchestra.run.smithi138:> ') 2020-11-16T22:23:33.007 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:33.008 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:33.008 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:33.008 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:33.012 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:33.015 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:33.035 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:33.057 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.070+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.062 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:23:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:23:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:23:33.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.154 INFO:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:23:33.158 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.172+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.181 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.182 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.190 INFO:teuthology.orchestra.run.smithi138.stdout:1 2020-11-16T22:23:33.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.191 INFO:teuthology.orchestra.run.smithi138:> route 2020-11-16T22:23:33.192 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:33.256 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2020-11-16T22:23:33.256 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:23:33.256 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 102 0 0 enp3s0f1 2020-11-16T22:23:33.257 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 102 0 0 enp3s0f1 2020-11-16T22:23:33.257 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:23:33.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.257 INFO:teuthology.orchestra.run.smithi138:> (cd / && exec sudo bash -c ' 2020-11-16T22:23:33.258 INFO:teuthology.orchestra.run.smithi138:> set -e 2020-11-16T22:23:33.258 INFO:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:23:33.258 INFO:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:23:33.258 INFO:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:23:33.259 INFO:teuthology.orchestra.run.smithi138:> ') 2020-11-16T22:23:33.262 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:33.275+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:33.316 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.316 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.325 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:33.362 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.376+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.463 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:33.477+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:23:33.480 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.480 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.487 INFO:teuthology.orchestra.run.smithi138:> ip netns list 2020-11-16T22:23:33.490 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:33.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.542 INFO:teuthology.orchestra.run.smithi138:> ip netns list-id 2020-11-16T22:23:33.563 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.577+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.596 INFO:teuthology.orchestra.run.smithi138:> (cd / && exec sudo bash -c ' 2020-11-16T22:23:33.596 INFO:teuthology.orchestra.run.smithi138:> set -e 2020-11-16T22:23:33.596 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-16T22:23:33.596 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2020-11-16T22:23:33.597 INFO:teuthology.orchestra.run.smithi138:> ') 2020-11-16T22:23:33.614 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.615 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.624 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:33.664 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:33.677+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:33.692 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-11-16T22:23:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.693 INFO:teuthology.orchestra.run.smithi138:> (cd / && exec sudo bash -c ' 2020-11-16T22:23:33.693 INFO:teuthology.orchestra.run.smithi138:> set -e 2020-11-16T22:23:33.694 INFO:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2020-11-16T22:23:33.694 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-16T22:23:33.694 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2020-11-16T22:23:33.694 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2020-11-16T22:23:33.695 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2020-11-16T22:23:33.695 INFO:teuthology.orchestra.run.smithi138:> ') 2020-11-16T22:23:33.748 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.748 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.758 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:33.803 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.816+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.865 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:33.878+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:23:33.894 INFO:teuthology.orchestra.run.smithi138:> (cd / && exec sudo bash -c ' 2020-11-16T22:23:33.894 INFO:teuthology.orchestra.run.smithi138:> set -e 2020-11-16T22:23:33.895 INFO:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2020-11-16T22:23:33.895 INFO:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2020-11-16T22:23:33.895 INFO:teuthology.orchestra.run.smithi138:> ') 2020-11-16T22:23:33.900 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:33.901 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:33.910 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:33.993 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:34.006+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:33.994 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:23:34.067 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:34.081+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:34.076 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.076 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.084 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:34.168 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:34.182+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:34.249 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.249 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.258 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:34.269 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:34.283+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:34.293 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:34.294 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":18,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:23:33.476789+0000","last_up_change":"2020-11-16T22:23:09.075128+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5801984,"flags_set":["noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":20752},{"type":"v1","addr":"172.21.15.173:6801","nonce":20752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":20752},{"type":"v1","addr":"172.21.15.173:6803","nonce":20752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":20752},{"type":"v1","addr":"172.21.15.173:6807","nonce":20752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":20752},{"type":"v1","addr":"172.21.15.173:6805","nonce":20752}]},"public_addr":"172.21.15.173:6801/20752","cluster_addr":"172.21.15.173:6803/20752","heartbeat_back_addr":"172.21.15.173:6807/20752","heartbeat_front_addr":"172.21.15.173:6805/20752","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":20834},{"type":"v1","addr":"172.21.15.122:6801","nonce":20834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":20834},{"type":"v1","addr":"172.21.15.122:6803","nonce":20834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":20834},{"type":"v1","addr":"172.21.15.122:6807","nonce":20834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":20834},{"type":"v1","addr":"172.21.15.122:6805","nonce":20834}]},"public_addr":"172.21.15.122:6801/20834","cluster_addr":"172.21.15.122:6803/20834","heartbeat_back_addr":"172.21.15.122:6807/20834","heartbeat_front_addr":"172.21.15.122:6805/20834","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:23:34.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:23:34.307 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:23:34.370 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:34.384+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:34.395 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.395 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.404 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:34.471 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:34.484+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:34.530 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.530 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.539 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:34.580 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:34.594+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:34.602 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2020-11-16T22:23:34.614 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:23:34.672 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:34.686+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:34.705 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.705 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.714 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:34.773 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:34.787+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:34.851 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:34.851 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:34.860 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:34.901 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:34.915+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:34.904 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:34.917 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:23:34.974 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:34.988+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:35.025 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:35.026 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:35.035 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:35.074 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:23:35.075 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:35.089+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:35.176 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:35.189+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:35.203 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:35.203 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:35.212 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:35.217 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:35.217 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:35.230 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:23:35.237 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:35.258 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:35.259 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:23:35.260 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:23:35.260 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:23:35.276 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:35.289+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:35.347 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:35.348 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:35.348 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:23:35.348 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:35.357 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:35.398 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:35.411+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:35.478 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:35.492+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:35.485 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:23:35.524 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:35.524 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:35.524 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:23:35.525 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:23:35.525 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:35.525 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 8 2020-11-16T22:23:35.531 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:35.537 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:23:35.537 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:35.580 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:35.593+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:35.657 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.425475904999999, 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:23:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:23:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:23:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.581878566, 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:23:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:23:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:23:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:23:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.470414714, 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:23:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:23:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:23:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.161546389000002, 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:23:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:23:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:23:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.161368651, 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:23:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:23:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:23:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.161188532000001, 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:23:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:23:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:23:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.161026271000001, 2020-11-16T22:23:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:23:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:23:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:23:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:23:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160895033999999, 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:23:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:23:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:23:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160723106999999, 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:23:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:23:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:23:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160529102999998, 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:23:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:23:35.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:23:35.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160328242999999, 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:35.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:23:35.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160160565999998, 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:23:35.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:23:35.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:23:35.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:23:35.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.160003230000001, 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:35.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:23:35.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:35.788 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.159873261000001, 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:23:35.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:23:35.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:23:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.156668318000001, 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:23:35.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:35.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:23:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.803 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.156555161, 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:23:35.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:23:35.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:23:35.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:35.812 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.156495871000001, 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:23:35.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:23:35.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:23:35.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.820 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.156116997000002, 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:35.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.155897489000001, 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:23:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:23:35.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.835 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.155854310999999, 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:23:35.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:23:35.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:23:35.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:35.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:35.843 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:35.845 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:35.845 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:35.858+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:35.930 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:35.930 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":4,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:23:17.499573+0000","modified":"2020-11-16T22:23:18.502988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4212":{"gid":4212,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.138:6803/1652701670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1652701670},{"type":"v1","addr":"172.21.15.138:6803","nonce":1652701670}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:23:35.932 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 4 2020-11-16T22:23:35.945 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:35.959+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:35.982 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:35.982 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:35.983 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:35.983 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:35.992 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:36.043 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 4, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:23:17.499573+0000', 'modified': '2020-11-16T22:23:18.502988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4212}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4212': {'gid': 4212, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.138:6803/1652701670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 1652701670}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 1652701670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:23:36.043 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:23:36.044 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:23:36.074 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.088+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.147 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.161+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.156 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:36.157 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:36.157 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:23:36.157 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:23:36.157 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:36.166 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:36.180 INFO:teuthology.orchestra.run.smithi138.stdout:{"cluster_fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","whoami":0,"id":4212,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.683286103,"mdsmap_epoch":4,"osdmap_epoch":18,"osdmap_epoch_barrier":16,"uptime":21.987872822} 2020-11-16T22:23:36.181 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", "fs_name": "cephfs", "id": 4212, "mdsmap_epoch": 4, "osdmap_epoch": 18, "osdmap_epoch_barrier": 16, "rank_uptime": 18.683286103, "state": "up:active", "uptime": 21.987872822, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:23:36.181 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-16T22:23:36.182 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-16T22:23:36.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:36.182 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p -v /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:23:36.196 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:23:36.196 INFO:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2020-11-16T22:23:36.249 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.263+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.290 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-16T22:23:36.290 INFO:teuthology.orchestra.run.smithi138:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-16T22:23:36.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:36.318 INFO:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2020-11-16T22:23:36.333 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:36.334 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:36.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:36.334 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:36.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.334 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.838001390999999, 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:36.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:36.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:36.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:36.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:36.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837889184000002, 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:36.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837798945999999, 2020-11-16T22:23:36.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:36.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:36.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:36.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:36.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.369 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837682303000001, 2020-11-16T22:23:36.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:36.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:36.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:36.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:36.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:36.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:36.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837533934, 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:36.382 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:36.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:36.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:36.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:36.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837344564999999, 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.394 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:36.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:36.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:36.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:36.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.404 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.837130642999998, 2020-11-16T22:23:36.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:36.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:36.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:36.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:36.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:36.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.416 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.836931742000001, 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:36.417 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:36.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:36.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:36.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:36.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.427 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.836750505000001, 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:36.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:30 3.3 e16/14 3:cd5a64a3:::100.00000000:head v 16'1, mlcod=0'0)", 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509778+0000", 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.83634528, 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0032173760000000001, 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.438 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509778+0000", 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509778+0000", 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1612000000000004e-05 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509859+0000", 2020-11-16T22:23:36.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7900000000000005e-06 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509868+0000", 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4881e-05 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509883+0000", 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7470000000000005e-06 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509891+0000", 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6536000000000002e-05 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509947+0000", 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5075e-05 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509972+0000", 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029641670000000002 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512936+0000", 2020-11-16T22:23:36.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8567999999999999e-05 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512995+0000", 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0032173760000000001 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.445 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.836123190999999, 2020-11-16T22:23:36.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:36.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:36.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:36.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:36.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:36.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:36.456 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.835825847999999, 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:36.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:36.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:36.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.835419036000001, 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.467 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:36.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.474 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.835319817999999, 2020-11-16T22:23:36.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:36.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:36.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:36.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:36.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.835124852, 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.486 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:36.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:36.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:36.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:36.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.495 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.83456378, 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:36.496 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:36.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:36.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.503 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.834405299, 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:36.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:36.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.834272387999999, 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:36.511 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:36.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:36.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:36.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:27 3.3 e16/14 3:d0630e4c:::mds0_sessionmap:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511955+0000", 2020-11-16T22:23:36.518 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.834168271999999, 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034734129999999998, 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511955+0000", 2020-11-16T22:23:36.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511955+0000", 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1791000000000004e-05 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6199999999999994e-06 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512036+0000", 2020-11-16T22:23:36.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4905999999999999e-05 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512051+0000", 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2090000000000002e-06 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512057+0000", 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00062723900000000003 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512684+0000", 2020-11-16T22:23:36.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4692999999999999e-05 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512709+0000", 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025320830000000001 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515241+0000", 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018687200000000001 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515428+0000", 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034734129999999998 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.526 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.834103778999999, 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:36.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:36.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:36.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:36.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.533 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.534 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.534 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:36.534 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:36.534 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:36.536 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:23:36.536 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:36.549+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:36.537 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:36.589 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-16T22:23:36.590 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --id 0 /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:23:36.632 INFO:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2020-11-16T22:23:36.636 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:36.650+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:36.659 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-16T22:23:36.659 INFO:teuthology.orchestra.run.smithi138:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-16T22:23:36.674 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:36.674 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:36.674 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:36.674 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:36.683 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-16T22:23:36.727 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-16T22:23:36.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:36.728 INFO:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2020-11-16T22:23:36.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2020-11-16T22:23:36.745+0000 7fb2c0f8e200 -1 init, newargv = 0x55ba731f3dd0 newargc=15 2020-11-16T22:23:36.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[23770]: starting ceph client 2020-11-16T22:23:36.736 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.750+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[23770]: starting fuse 2020-11-16T22:23:36.746 INFO:teuthology.orchestra.run.smithi138.stdout:47 2020-11-16T22:23:36.746 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-16T22:23:36.746 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:36.747 INFO:teuthology.orchestra.run.smithi138:> import glob 2020-11-16T22:23:36.747 INFO:teuthology.orchestra.run.smithi138:> import re 2020-11-16T22:23:36.747 INFO:teuthology.orchestra.run.smithi138:> import os 2020-11-16T22:23:36.747 INFO:teuthology.orchestra.run.smithi138:> import subprocess 2020-11-16T22:23:36.748 INFO:teuthology.orchestra.run.smithi138:> 2020-11-16T22:23:36.748 INFO:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2020-11-16T22:23:36.748 INFO:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-16T22:23:36.748 INFO:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2020-11-16T22:23:36.748 INFO:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-11-16T22:23:36.749 INFO:teuthology.orchestra.run.smithi138:> 2020-11-16T22:23:36.749 INFO:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2020-11-16T22:23:36.749 INFO:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2020-11-16T22:23:36.749 INFO:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2020-11-16T22:23:36.750 INFO:teuthology.orchestra.run.smithi138:> return files[0] 2020-11-16T22:23:36.750 INFO:teuthology.orchestra.run.smithi138:> 2020-11-16T22:23:36.750 INFO:teuthology.orchestra.run.smithi138:> for f in files: 2020-11-16T22:23:36.750 INFO:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-16T22:23:36.751 INFO:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-16T22:23:36.751 INFO:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-11-16T22:23:36.751 INFO:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2020-11-16T22:23:36.751 INFO:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2020-11-16T22:23:36.751 INFO:teuthology.orchestra.run.smithi138:> return f 2020-11-16T22:23:36.752 INFO:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-16T22:23:36.752 INFO:teuthology.orchestra.run.smithi138:> 2020-11-16T22:23:36.752 INFO:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2020-11-16T22:23:36.752 INFO:teuthology.orchestra.run.smithi138:> ' 2020-11-16T22:23:36.837 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.851+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.847 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:23:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:23:36.848 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:36.856 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:36.869 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.23770.asok 2020-11-16T22:23:36.937 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:36.951+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:36.982 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:36.982 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:23:36.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:23:36.982 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 26.906045675000001, 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:36.983 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:36.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:36.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:36.990 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 26.794615994000001, 2020-11-16T22:23:36.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:36.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:23:36.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:23:36.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:23:36.998 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:36.999 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.486179443000001, 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:37.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:23:37.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:23:37.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:23:37.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:23:37.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:37.009 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.486064143, 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:23:37.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:23:37.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.019 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485955383, 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:23:37.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:23:37.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.029 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485873034000001, 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:23:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:23:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:23:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:23:37.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.039 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485785226000001, 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:37.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:23:37.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:23:37.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:23:37.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485690687000002, 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485604501000001, 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.060 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:23:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:23:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:23:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.485504116000001, 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:23:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:23:37.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:23:37.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:37.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:23:37.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.480390795000002, 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.080 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:23:37.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:23:37.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:23:37.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.480284621999999, 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:23:37.088 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.095 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.480110612000001, 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:23:37.096 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:23:37.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.103 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.480014333, 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:23:37.104 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:23:37.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:23:37.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:23:37.111 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.479826631000002, 2020-11-16T22:23:37.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:23:37.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:23:37.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.479634677, 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.119 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:23:37.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:23:37.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:23:37.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.479472113, 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:23:37.127 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:23:37.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:23:37.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:23:37.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.134 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.479296251000001, 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:23:37.135 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:23:37.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:23:37.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:23:37.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:37.143 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.479128205999999, 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:23:37.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:23:37.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:23:37.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:37.151 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.478887075999999, 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:37.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:23:37.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:37.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:23:37.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:37.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:37.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:37.159 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:37.161 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:37.174+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:37.161 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23770.asok 2020-11-16T22:23:37.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:37.162 INFO:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23770.asok status 2020-11-16T22:23:37.163 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:37.262 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:37.275+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:37.271 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:37.299 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:23:37.299 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:37.308 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:23:37.319 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2020-11-16T22:23:37.319 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "b637909a9153a419511f943e4eee7800ad3bbb99", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 16.0.0-7198-gb637909a (b637909a9153a419511f943e4eee7800ad3bbb99) pacific (dev)", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "23770", 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2020-11-16T22:23:37.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4471, 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4471 2020-11-16T22:23:37.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2424814314 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:23:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2424814314 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4471 192.168.0.1:0/2424814314", 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/2424814314", 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2020-11-16T22:23:37.323 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 4, 2020-11-16T22:23:37.324 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 19, 2020-11-16T22:23:37.324 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2020-11-16T22:23:37.324 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false 2020-11-16T22:23:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:23:37.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:37.329 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-16T22:23:37.330 INFO:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:23:37.344 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2020-11-16T22:23:37.344 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-16T22:23:37.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:37.345 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:23:37.362 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:37.376+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:37.412 INFO:teuthology.run_tasks:Running task workunit... 2020-11-16T22:23:37.426 INFO:tasks.workunit:Pulling workunits from ref 6441771be52a0f47b9e118f8bca335f29dfda438 2020-11-16T22:23:37.427 INFO:tasks.workunit:Making a separate scratch dir for every client... 2020-11-16T22:23:37.427 INFO:tasks.workunit:timeout=6h 2020-11-16T22:23:37.428 INFO:tasks.workunit:cleanup=True 2020-11-16T22:23:37.428 INFO:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.0 2020-11-16T22:23:37.429 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:37.432 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:37.432 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:37.432 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:23:37.432 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:23:37.432 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:37.438 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:37.441 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:37.442 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.0 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 2fh/47d Inode: 1 Links: 2 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fusefs_t:s0 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2020-11-16 22:23:17.503492159 +0000 2020-11-16T22:23:37.443 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2020-11-16 22:23:37.425539576 +0000 2020-11-16T22:23:37.444 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2020-11-16T22:23:37.444 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2020-11-16T22:23:37.444 INFO:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2020-11-16T22:23:37.459 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:37.460 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:23:37.461 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:23:37.461 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:23:37.486 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:37.499+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:37.490 INFO:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6441771be52a0f47b9e118f8bca335f29dfda438 2020-11-16T22:23:37.564 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:37.577+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:37.565 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:37.566 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:37.575 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:37.604 INFO:tasks.workunit.client.0.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2020-11-16T22:23:37.665 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:37.679+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:37.721 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:37.765 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:37.779+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:37.866 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:37.879+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:37.897 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:37.967 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:37.981+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:38.009 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:23:38.009 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:23:38.010 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:23:38.062 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.062 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.068 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:38.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:38.071 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:38.168 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:38.182+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:38.207 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.207 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.216 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:38.270 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.284+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:38.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:23:38.311 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:23:38.322 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2020-11-16T22:23:37.629342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":15324,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":14784,"kb_avail":283100196,"statfs":{"total":289910292480,"available":289894600704,"internally_reserved":0,"allocated":528384,"data_stored":175960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15138816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001244"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.514910+0000","last_change":"2020-11-16T22:23:17.511331+0000","last_active":"2020-11-16T22:23:18.514910+0000","last_peered":"2020-11-16T22:23:18.514910+0000","last_clean":"2020-11-16T22:23:18.514910+0000","last_became_active":"2020-11-16T22:23:17.510901+0000","last_became_peered":"2020-11-16T22:23:17.510901+0000","last_unstale":"2020-11-16T22:23:18.514910+0000","last_undegraded":"2020-11-16T22:23:18.514910+0000","last_fullsized":"2020-11-16T22:23:18.514910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530426+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:23:18.530426+0000","last_peered":"2020-11-16T22:23:18.530426+0000","last_clean":"2020-11-16T22:23:18.530426+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:23:18.530426+0000","last_undegraded":"2020-11-16T22:23:18.530426+0000","last_fullsized":"2020-11-16T22:23:18.530426+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"17","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515107+0000","last_change":"2020-11-16T22:23:17.139458+0000","last_active":"2020-11-16T22:23:18.515107+0000","last_peered":"2020-11-16T22:23:18.515107+0000","last_clean":"2020-11-16T22:23:18.515107+0000","last_became_active":"2020-11-16T22:23:17.139100+0000","last_became_peered":"2020-11-16T22:23:17.139100+0000","last_unstale":"2020-11-16T22:23:18.515107+0000","last_undegraded":"2020-11-16T22:23:18.515107+0000","last_fullsized":"2020-11-16T22:23:18.515107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530518+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:23:18.530518+0000","last_peered":"2020-11-16T22:23:18.530518+0000","last_clean":"2020-11-16T22:23:18.530518+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:23:18.530518+0000","last_undegraded":"2020-11-16T22:23:18.530518+0000","last_fullsized":"2020-11-16T22:23:18.530518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515076+0000","last_change":"2020-11-16T22:23:14.110329+0000","last_active":"2020-11-16T22:23:18.515076+0000","last_peered":"2020-11-16T22:23:18.515076+0000","last_clean":"2020-11-16T22:23:18.515076+0000","last_became_active":"2020-11-16T22:23:14.110005+0000","last_became_peered":"2020-11-16T22:23:14.110005+0000","last_unstale":"2020-11-16T22:23:18.515076+0000","last_undegraded":"2020-11-16T22:23:18.515076+0000","last_fullsized":"2020-11-16T22:23:18.515076+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"16","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515187+0000","last_change":"2020-11-16T22:23:17.139215+0000","last_active":"2020-11-16T22:23:18.515187+0000","last_peered":"2020-11-16T22:23:18.515187+0000","last_clean":"2020-11-16T22:23:18.515187+0000","last_became_active":"2020-11-16T22:23:17.138850+0000","last_became_peered":"2020-11-16T22:23:17.138850+0000","last_unstale":"2020-11-16T22:23:18.515187+0000","last_undegraded":"2020-11-16T22:23:18.515187+0000","last_fullsized":"2020-11-16T22:23:18.515187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530581+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:23:18.530581+0000","last_peered":"2020-11-16T22:23:18.530581+0000","last_clean":"2020-11-16T22:23:18.530581+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:23:18.530581+0000","last_undegraded":"2020-11-16T22:23:18.530581+0000","last_fullsized":"2020-11-16T22:23:18.530581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530332+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:23:18.530332+0000","last_peered":"2020-11-16T22:23:18.530332+0000","last_clean":"2020-11-16T22:23:18.530332+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:23:18.530332+0000","last_undegraded":"2020-11-16T22:23:18.530332+0000","last_fullsized":"2020-11-16T22:23:18.530332+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"19","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.514824+0000","last_change":"2020-11-16T22:23:17.137728+0000","last_active":"2020-11-16T22:23:18.514824+0000","last_peered":"2020-11-16T22:23:18.514824+0000","last_clean":"2020-11-16T22:23:18.514824+0000","last_became_active":"2020-11-16T22:23:17.137370+0000","last_became_peered":"2020-11-16T22:23:17.137370+0000","last_unstale":"2020-11-16T22:23:18.514824+0000","last_undegraded":"2020-11-16T22:23:18.514824+0000","last_fullsized":"2020-11-16T22:23:18.514824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.514979+0000","last_change":"2020-11-16T22:23:17.513110+0000","last_active":"2020-11-16T22:23:18.514979+0000","last_peered":"2020-11-16T22:23:18.514979+0000","last_clean":"2020-11-16T22:23:18.514979+0000","last_became_active":"2020-11-16T22:23:17.512213+0000","last_became_peered":"2020-11-16T22:23:17.512213+0000","last_unstale":"2020-11-16T22:23:18.514979+0000","last_undegraded":"2020-11-16T22:23:18.514979+0000","last_fullsized":"2020-11-16T22:23:18.514979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515039+0000","last_change":"2020-11-16T22:23:14.110609+0000","last_active":"2020-11-16T22:23:18.515039+0000","last_peered":"2020-11-16T22:23:18.515039+0000","last_clean":"2020-11-16T22:23:18.515039+0000","last_became_active":"2020-11-16T22:23:14.110218+0000","last_became_peered":"2020-11-16T22:23:14.110218+0000","last_unstale":"2020-11-16T22:23:18.515039+0000","last_undegraded":"2020-11-16T22:23:18.515039+0000","last_fullsized":"2020-11-16T22:23:18.515039+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"21","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515006+0000","last_change":"2020-11-16T22:23:10.087624+0000","last_active":"2020-11-16T22:23:18.515006+0000","last_peered":"2020-11-16T22:23:18.515006+0000","last_clean":"2020-11-16T22:23:18.515006+0000","last_became_active":"2020-11-16T22:23:10.087051+0000","last_became_peered":"2020-11-16T22:23:10.087051+0000","last_unstale":"2020-11-16T22:23:18.515006+0000","last_undegraded":"2020-11-16T22:23:18.515006+0000","last_fullsized":"2020-11-16T22:23:18.515006+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"12","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530329+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:23:18.530329+0000","last_peered":"2020-11-16T22:23:18.530329+0000","last_clean":"2020-11-16T22:23:18.530329+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:23:18.530329+0000","last_undegraded":"2020-11-16T22:23:18.530329+0000","last_fullsized":"2020-11-16T22:23:18.530329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.514983+0000","last_change":"2020-11-16T22:23:17.515386+0000","last_active":"2020-11-16T22:23:18.514983+0000","last_peered":"2020-11-16T22:23:18.514983+0000","last_clean":"2020-11-16T22:23:18.514983+0000","last_became_active":"2020-11-16T22:23:17.514774+0000","last_became_peered":"2020-11-16T22:23:17.514774+0000","last_unstale":"2020-11-16T22:23:18.514983+0000","last_undegraded":"2020-11-16T22:23:18.514983+0000","last_fullsized":"2020-11-16T22:23:18.514983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515012+0000","last_change":"2020-11-16T22:23:14.107348+0000","last_active":"2020-11-16T22:23:18.515012+0000","last_peered":"2020-11-16T22:23:18.515012+0000","last_clean":"2020-11-16T22:23:18.515012+0000","last_became_active":"2020-11-16T22:23:14.106953+0000","last_became_peered":"2020-11-16T22:23:14.106953+0000","last_unstale":"2020-11-16T22:23:18.515012+0000","last_undegraded":"2020-11-16T22:23:18.515012+0000","last_fullsized":"2020-11-16T22:23:18.515012+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"11","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530316+0000","last_change":"2020-11-16T22:23:17.138337+0000","last_active":"2020-11-16T22:23:18.530316+0000","last_peered":"2020-11-16T22:23:18.530316+0000","last_clean":"2020-11-16T22:23:18.530316+0000","last_became_active":"2020-11-16T22:23:17.137214+0000","last_became_peered":"2020-11-16T22:23:17.137214+0000","last_unstale":"2020-11-16T22:23:18.530316+0000","last_undegraded":"2020-11-16T22:23:18.530316+0000","last_fullsized":"2020-11-16T22:23:18.530316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.514913+0000","last_change":"2020-11-16T22:23:17.515838+0000","last_active":"2020-11-16T22:23:18.514913+0000","last_peered":"2020-11-16T22:23:18.514913+0000","last_clean":"2020-11-16T22:23:18.514913+0000","last_became_active":"2020-11-16T22:23:17.515353+0000","last_became_peered":"2020-11-16T22:23:17.515353+0000","last_unstale":"2020-11-16T22:23:18.514913+0000","last_undegraded":"2020-11-16T22:23:18.514913+0000","last_fullsized":"2020-11-16T22:23:18.514913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515032+0000","last_change":"2020-11-16T22:23:14.112895+0000","last_active":"2020-11-16T22:23:18.515032+0000","last_peered":"2020-11-16T22:23:18.515032+0000","last_clean":"2020-11-16T22:23:18.515032+0000","last_became_active":"2020-11-16T22:23:14.112524+0000","last_became_peered":"2020-11-16T22:23:14.112524+0000","last_unstale":"2020-11-16T22:23:18.515032+0000","last_undegraded":"2020-11-16T22:23:18.515032+0000","last_fullsized":"2020-11-16T22:23:18.515032+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"16'8","reported_seq":"19","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530666+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:23:18.530666+0000","last_peered":"2020-11-16T22:23:18.530666+0000","last_clean":"2020-11-16T22:23:18.530666+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:23:18.530666+0000","last_undegraded":"2020-11-16T22:23:18.530666+0000","last_fullsized":"2020-11-16T22:23:18.530666+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530332+0000","last_change":"2020-11-16T22:23:17.513578+0000","last_active":"2020-11-16T22:23:18.530332+0000","last_peered":"2020-11-16T22:23:18.530332+0000","last_clean":"2020-11-16T22:23:18.530332+0000","last_became_active":"2020-11-16T22:23:17.511994+0000","last_became_peered":"2020-11-16T22:23:17.511994+0000","last_unstale":"2020-11-16T22:23:18.530332+0000","last_undegraded":"2020-11-16T22:23:18.530332+0000","last_fullsized":"2020-11-16T22:23:18.530332+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"17","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530612+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:23:18.530612+0000","last_peered":"2020-11-16T22:23:18.530612+0000","last_clean":"2020-11-16T22:23:18.530612+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:23:18.530612+0000","last_undegraded":"2020-11-16T22:23:18.530612+0000","last_fullsized":"2020-11-16T22:23:18.530612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.530532+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:23:18.530532+0000","last_peered":"2020-11-16T22:23:18.530532+0000","last_clean":"2020-11-16T22:23:18.530532+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:23:18.530532+0000","last_undegraded":"2020-11-16T22:23:18.530532+0000","last_fullsized":"2020-11-16T22:23:18.530532+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"10","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515112+0000","last_change":"2020-11-16T22:23:17.511513+0000","last_active":"2020-11-16T22:23:18.515112+0000","last_peered":"2020-11-16T22:23:18.515112+0000","last_clean":"2020-11-16T22:23:18.515112+0000","last_became_active":"2020-11-16T22:23:17.511073+0000","last_became_peered":"2020-11-16T22:23:17.511073+0000","last_unstale":"2020-11-16T22:23:18.515112+0000","last_undegraded":"2020-11-16T22:23:18.515112+0000","last_fullsized":"2020-11-16T22:23:18.515112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:18.515043+0000","last_change":"2020-11-16T22:23:14.107363+0000","last_active":"2020-11-16T22:23:18.515043+0000","last_peered":"2020-11-16T22:23:18.515043+0000","last_clean":"2020-11-16T22:23:18.515043+0000","last_became_active":"2020-11-16T22:23:14.106918+0000","last_became_peered":"2020-11-16T22:23:14.106918+0000","last_unstale":"2020-11-16T22:23:18.515043+0000","last_undegraded":"2020-11-16T22:23:18.515043+0000","last_fullsized":"2020-11-16T22:23:18.515043+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"17'4","reported_seq":"15","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T22:23:19.202001+0000","last_change":"2020-11-16T22:23:17.137398+0000","last_active":"2020-11-16T22:23:19.202001+0000","last_peered":"2020-11-16T22:23:19.202001+0000","last_clean":"2020-11-16T22:23:19.202001+0000","last_became_active":"2020-11-16T22:23:17.136876+0000","last_became_peered":"2020-11-16T22:23:17.136876+0000","last_unstale":"2020-11-16T22:23:19.202001+0000","last_undegraded":"2020-11-16T22:23:19.202001+0000","last_fullsized":"2020-11-16T22:23:19.202001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":57504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":184320,"data_stored":59036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":176128,"data_stored":59420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}]}} 2020-11-16T22:23:38.340 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.341 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.350 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:38.371 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:38.384+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:38.472 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.485+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:38.474 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.474 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.483 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:38.537 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:23:38.572 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.585+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:38.624 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.624 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.634 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:38.675 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.688+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:38.773 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:38.787+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:38.800 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.800 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.809 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:38.874 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.887+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:38.933 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:38.933 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:38.942 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:38.983 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:38.996+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:39.076 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:39.090+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:39.078 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:39.079 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:39.089 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:39.177 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:39.190+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:39.214 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:39.215 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:39.224 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:39.277 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:39.290+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:39.348 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:39.348 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:39.357 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:39.401 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:39.415+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:39.471 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:23:39.478 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:39.492+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:39.493 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:39.493 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:39.502 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:39.586 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:39.600+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:39.614 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:39.635 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:39.655 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:23:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:23:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:23:39.669 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:39.669 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:39.669 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:23:39.669 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:23:39.669 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:39.678 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:39.719 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:39.733+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:39.782 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:39.795+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:23:39.844 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:23:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:23:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.768351565, 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.852 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.656921883999999, 2020-11-16T22:23:39.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:39.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:23:39.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:23:39.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:23:39.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:39.861 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.348485332999999, 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:39.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:23:39.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:23:39.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:23:39.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:23:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.348370032999998, 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:23:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:23:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.881 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.348261272999999, 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:23:39.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:23:39.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:23:39.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:23:39.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.891 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.348178923999999, 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:23:39.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:23:39.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:23:39.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:23:39.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.901 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.348091115999999, 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:39.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:23:39.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:23:39.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:23:39.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.347996577, 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:39.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:23:39.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:23:39.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:23:39.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.347910390999999, 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:23:39.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.347810006, 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.932 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:23:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.342696685, 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:23:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:23:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:23:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.342590512000001, 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:23:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:23:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:23:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.342416501999999, 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:23:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:23:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:23:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.342320223000002, 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:23:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.342132521, 2020-11-16T22:23:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:23:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:23:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.341940567000002, 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:23:39.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:23:39.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:23:39.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.988 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.341778003000002, 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.989 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:23:39.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:23:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:23:39.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:39.996 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.341602140999999, 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:23:39.997 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:23:39.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:23:40.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:23:40.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:40.004 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.341434096, 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:23:40.005 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:40.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:23:40.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.012 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.341192966000001, 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:23:40.013 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:40.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:40.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:40.021 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:40.022 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:40.023 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:40.036+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:40.123 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:40.137+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:40.158 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:40.158 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:40.158 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:23:40.158 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:40.167 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:40.224 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:40.237+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:40.291 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:40.292 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:40.292 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:23:40.292 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:23:40.292 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:40.300 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:40.341 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:40.354+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:40.426 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:40.440+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:40.465 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:23:40.466 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.234079723000001, 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:40.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:40.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:23:40.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:40.475 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.390482383999998, 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:40.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:23:40.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:23:40.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:40.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:23:40.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.485 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.279018531999998, 2020-11-16T22:23:40.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.487 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:23:40.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:23:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:23:40.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:23:40.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.970150207, 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:23:40.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:23:40.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:23:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:40.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969972468999998, 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:23:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:23:40.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:23:40.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:23:40.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:23:40.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969792349999999, 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:23:40.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:23:40.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:23:40.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969630088999999, 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:23:40.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:23:40.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:23:40.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:40.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:23:40.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:23:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969498852000001, 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:23:40.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:23:40.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:23:40.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:23:40.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969326925000001, 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:23:40.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:23:40.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:23:40.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:23:40.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.969132921, 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:23:40.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:23:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:23:40.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:23:40.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.968932061, 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:23:40.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:23:40.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:40.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:23:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.968764384, 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:23:40.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:23:40.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:23:40.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:40.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:23:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.968607047999999, 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:40.588 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:23:40.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:23:40.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:23:40.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.968477078999999, 2020-11-16T22:23:40.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:40.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:23:40.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:23:40.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.602 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.965272135999999, 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:40.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:23:40.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:23:40.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.610 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.965158979000002, 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:40.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:23:40.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:23:40.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:23:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:23:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.965099688999999, 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:23:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.964720815, 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:23:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.964501306999999, 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:23:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.964458129, 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:23:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:23:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:23:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:40.651 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:40.652 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:40.665+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:40.652 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:40.753 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:40.767+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:40.788 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:40.789 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:40.798 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:40.853 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:40.867+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:40.923 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:40.923 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:40.923 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:23:40.923 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:23:40.924 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:40.932 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:40.954 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:40.968+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:41.056 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:41.069+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:41.098 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:41.098 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.602575891000001, 2020-11-16T22:23:41.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:41.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:41.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:41.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:41.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:41.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:41.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.602463684, 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.111 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:41.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:41.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:41.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:41.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:41.121 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.602373446000001, 2020-11-16T22:23:41.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:41.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:41.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:41.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:41.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:41.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.133 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.602256803, 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:41.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:41.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:41.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:41.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:41.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.602108434000002, 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.145 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:41.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:41.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.601919065000001, 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.601705143, 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:41.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.601506241999999, 2020-11-16T22:23:41.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:41.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:41.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:41.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.188 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.601325005, 2020-11-16T22:23:41.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:41.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:41.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:41.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:41.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:41.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.600697691000001, 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:41.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:41.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:41.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:41.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:41.209 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.600400348000001, 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:41.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:41.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:41.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:41.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.599993535999999, 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:41.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:41.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:41.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:41.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:41.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:41.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:41.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:41.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:41.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:41.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:41.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.230 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.599894318, 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:41.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:41.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:41.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:41.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.240 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.599699351999998, 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:41.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:41.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:41.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:41.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.250 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.599138279999998, 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:41.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:41.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:41.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.257 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.598979798999999, 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.258 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.598846888000001, 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.598678279000001, 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4.3629745179999997, 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:23:41.280 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:23:41.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:23:41.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:41.291 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4.3628474910000001, 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:41.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:41.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:23:41.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:23:41.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:23:41.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:41.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:41.302 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:41.303 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:41.304 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:41.304 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:41.317+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:41.404 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:41.418+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:41.440 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:41.440 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:41.440 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:41.440 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:41.449 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:41.505 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:41.519+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:41.574 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:41.574 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:41.574 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:23:41.575 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:23:41.575 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:41.583 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:41.626 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:41.640+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:41.667 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:41.706 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:41.720+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:41.709 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:41.710 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.633814253000001, 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:23:41.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:23:41.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:23:41.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.718 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.522384572, 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:23:41.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:23:41.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:23:41.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.726 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213948021, 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:23:41.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:23:41.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:23:41.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.736 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213832720999999, 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:41.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:23:41.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:23:41.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:23:41.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213723960999999, 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:23:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:23:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:23:41.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:23:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213641612, 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:23:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:23:41.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:23:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:23:41.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.766 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213553804, 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:41.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:23:41.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:23:41.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:23:41.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.776 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213459265000001, 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:23:41.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:23:41.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:23:41.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.786 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213373079, 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:23:41.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:23:41.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:23:41.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:41.796 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.213272694, 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:23:41.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:41.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:23:41.805 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.208159373000001, 2020-11-16T22:23:41.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:41.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:23:41.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:23:41.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.813 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.208053199999998, 2020-11-16T22:23:41.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:23:41.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.821 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.20787919, 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:23:41.822 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:23:41.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:23:41.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.207782910999999, 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:23:41.830 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:23:41.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:23:41.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:23:41.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:23:41.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.207595209000001, 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:41.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:23:41.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:23:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.207403254999999, 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:23:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.207240690999999, 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:23:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:23:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:23:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.207064829, 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:23:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:23:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.206896784000001, 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:23:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:23:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.206655653999999, 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:23:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:23:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:41.885 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:41.887 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:41.887 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:41.888 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:41.888 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:23:41.888 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:23:41.888 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:23:41.889 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:41.889 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:41.901+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:41.987 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:42.001+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:42.023 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:23:42.031 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.031 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.040 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:42.088 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:42.103+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:42.165 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.165 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.174 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:42.189 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.203+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.290 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.304+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.298 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.298 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.307 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:42.391 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:42.405+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:42.444 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.445 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.454 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:42.497 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.511+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.579 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.579 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.588 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:42.629 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.643+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:23:42.693 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:42.707+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:42.752 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.752 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.761 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:42.795 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.808+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.895 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:42.909+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:42.897 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:42.897 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:42.906 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:42.996 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:43.010+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:43.030 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.031 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.040 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:43.123 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:43.137+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:43.198 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:43.212+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:43.206 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.206 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.215 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:43.299 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:43.313+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:43.324 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:23:43.324 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:23:43.324 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:23:43.324 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:23:43.324 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:23:43.325 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:23:43.349 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.349 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.358 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:43.400 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:43.414+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:43.449 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:43.449 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-11-16T22:23:43.449 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:43.481 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.481 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.490 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:43.501 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:43.515+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:43.601 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:43.615+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:43.614 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.615 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.624 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:43.702 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:43.716+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:43.759 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.759 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.769 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:43.809 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:43.823+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:43.889 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:23:43.904 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:43.918+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:43.932 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:43.932 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:43.942 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:44.005 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:44.019+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:44.043 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:44.063 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:44.069 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:44.070 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:44.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:44.070 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:44.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.070 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573973157000001, 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:44.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:44.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:44.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:44.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:44.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.57386095, 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:44.082 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:44.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:44.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:44.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:44.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:44.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:44.093 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573770712000002, 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:44.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573654069, 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:44.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:44.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:44.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:44.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.115 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573505699999998, 2020-11-16T22:23:44.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:44.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:44.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:44.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:44.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.127 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573316331000001, 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:44.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:44.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:44.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.573102409000001, 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.139 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:44.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:44.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:44.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:44.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:44.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.149 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.572903508, 2020-11-16T22:23:44.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:44.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:44.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.159 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.572722271, 2020-11-16T22:23:44.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:44.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:44.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.169 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.572094957000001, 2020-11-16T22:23:44.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:44.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:44.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:44.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:44.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.571797614000001, 2020-11-16T22:23:44.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:44.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:44.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:44.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:44.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:44.190 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.571390802, 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:44.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:44.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:44.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:44.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:44.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.571291584000001, 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.208 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.571096617999999, 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:44.209 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:44.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:44.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:44.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:44.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.570535545999999, 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:44.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.570377064999999, 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:44.227 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:44.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:44.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:44.233 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:44.234 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.570244154000001, 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:44.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:44.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:44.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.570075545000002, 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:44.242 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 7.334371784, 2020-11-16T22:23:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:23:44.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:23:44.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:23:44.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:23:44.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.260 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 7.3342447569999996, 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:44.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:23:44.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:23:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:23:44.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:23:44.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.272 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:44.272 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:44.272 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:44.273 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:44.274 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:23:44.274 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:23:44.274 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:23:44.274 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:44.287+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:44.275 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:44.374 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:44.388+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:44.407 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:44.407 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:44.416 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:44.499 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:44.513+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:44.576 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:44.590+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:44.581 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:44.581 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:44.581 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:23:44.582 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:23:44.582 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:44.591 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:44.677 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:44.691+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:44.687 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:23:44.757 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:23:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:23:44.757 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 34.681224391000001, 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:23:44.758 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:23:44.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:23:44.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 34.569794709999996, 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.767 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:23:44.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:23:44.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:23:44.774 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:44.775 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.261358159, 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:23:44.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:23:44.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:23:44.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:23:44.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.261242858999999, 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:23:44.786 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:23:44.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:23:44.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:23:44.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.261134099, 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:23:44.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:23:44.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.807 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.26105175, 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:23:44.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:23:44.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.818 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.260963942, 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:23:44.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:23:44.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:23:44.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:23:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.260869403000001, 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:23:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:23:44.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:23:44.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:23:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.260783217, 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.840 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:44.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:23:44.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:23:44.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:23:44.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:44.850 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.260682832000001, 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:44.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:23:44.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:23:44.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.860 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.255569511000001, 2020-11-16T22:23:44.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:44.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:23:44.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:23:44.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.255463337999998, 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.870 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:23:44.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:23:44.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:23:44.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:44.878 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.255289328, 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:44.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:23:44.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.255193048999999, 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:23:44.887 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:23:44.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:23:44.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.895 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.255005347000001, 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:23:44.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:23:44.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.254813392999999, 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.903 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:23:44.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:23:44.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:44.911 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.254650828999999, 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:23:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:23:44.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:23:44.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.254474967, 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.920 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:23:44.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:23:44.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:23:44.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.254306922000001, 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:44.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:23:44.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:23:44.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.254065791999999, 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:23:44.937 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:44.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:44.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:44.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:23:44.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:44.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:44.945 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:44.946 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:44.947 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:44.947 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:44.961+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:45.047 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:45.061+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:45.081 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:23:45.082 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:45.091 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:45.149 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:45.162+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:45.214 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:45.215 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:45.215 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:23:45.215 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:23:45.215 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:45.223 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:45.248 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:45.262+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:45.349 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:45.350 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:23:45.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:23:45.350 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:23:45.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.350 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.117808295000003, 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:45.351 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:45.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:23:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:23:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.274210955999997, 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:23:45.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:23:45.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:23:45.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:23:45.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:45.370 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.162747103999997, 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:23:45.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:23:45.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:23:45.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:23:45.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:23:45.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853878778999999, 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:23:45.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:23:45.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:23:45.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853701041000001, 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:23:45.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:23:45.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.400 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853520922000001, 2020-11-16T22:23:45.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:45.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:45.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:23:45.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:23:45.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:23:45.410 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:45.411 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853358661000001, 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:45.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:23:45.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:23:45.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:23:45.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:23:45.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:45.421 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853227424, 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:23:45.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:23:45.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.431 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.853055497, 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:23:45.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:23:45.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:23:45.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:23:45.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.441 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.852861492999999, 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:45.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:23:45.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:23:45.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:23:45.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.852660632999999, 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:23:45.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:23:45.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:23:45.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:23:45.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.852492955999999, 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:23:45.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:23:45.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:23:45.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:23:45.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.852335620000002, 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:23:45.472 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:23:45.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:45.479 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.852205650999998, 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:23:45.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.849000707999998, 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.487 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:23:45.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:23:45.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:23:45.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.848887551000001, 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.495 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:45.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:23:45.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.848828261000001, 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.503 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:23:45.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:23:45.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:23:45.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.848449386999999, 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.511 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:23:45.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:23:45.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:23:45.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.848229879000002, 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.519 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:23:45.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:23:45.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.525 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.848186700999999, 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:45.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:45.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:23:45.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:23:45.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:45.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:45.533 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:45.535 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:45.363+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:45.535 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:45.549+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:45.535 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:45.635 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:45.649+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:45.673 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:45.673 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:45.673 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:45.674 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:45.682 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:45.735 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:45.749+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:45.836 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:45.850+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:23:45.847 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:45.847 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:45.848 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:23:45.848 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:23:45.848 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:45.856 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:45.937 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:45.951+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:46.023 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:46.023 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527854113, 2020-11-16T22:23:46.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:46.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:46.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:46.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:46.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:46.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.035 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527741905999999, 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.036 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527651668000001, 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527535024999999, 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527386656000001, 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.527197287, 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.526983365, 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.526784463999999, 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.526603226999999, 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.525975913, 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.52567857, 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.147 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.525271757999999, 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:46.148 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:46.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:46.155 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.52517254, 2020-11-16T22:23:46.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:46.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:46.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.524977574000001, 2020-11-16T22:23:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:46.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:46.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:46.175 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.524416502000001, 2020-11-16T22:23:46.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:46.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:46.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:46.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.524258021000001, 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:46.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.190 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.52412511, 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:46.191 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:46.198 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.523956501000001, 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:46.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:46.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 9.2882527400000008, 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.206 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:23:46.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:23:46.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:23:46.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:23:46.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:23:46.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:46.217 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 9.2881257129999995, 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:46.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:23:46.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:23:46.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:23:46.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:23:46.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:23:46.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:46.229 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:46.229 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:46.230 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.244+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.230 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:46.275 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:46.331 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:46.345+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:46.367 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:46.368 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:46.368 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:46.368 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:46.377 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:46.427 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:46.432 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.446+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.446 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:46.466 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:46.467 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:23:46.467 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:23:46.467 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:23:46.533 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.547+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.542 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:46.542 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:46.551 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:46.634 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.648+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.675 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:46.675 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:46.684 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:46.735 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:46.749+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:46.820 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:46.820 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:46.829 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:46.836 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.850+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.937 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:46.951+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:46.993 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:46.993 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.002 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:47.037 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:47.051+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:47.127 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.127 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.135 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:47.138 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:47.152+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:47.239 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:47.253+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:47.273 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.273 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.283 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:47.339 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:47.354+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:47.408 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.408 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.417 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:47.439 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:47.454+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:47.540 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:47.554+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:47.542 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.542 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.551 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:47.641 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:47.655+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:47.688 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.688 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.697 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:47.741 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:47.756+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:47.820 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.821 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.830 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:47.843 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:47.857+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:47.856 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:23:47.943 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:47.958+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:47.954 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:47.954 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:47.963 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:48.044 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:48.058+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:48.099 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:48.099 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:48.108 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:48.145 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:48.159+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:48.231 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:48.232 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:48.240 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:48.246 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:48.261+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:48.347 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:48.361+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:48.365 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:48.365 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:48.374 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:48.448 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:48.462+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:48.458 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:23:48.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:23:48.459 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2020-11-16T22:23:48.459 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2020-11-16T22:23:48.477 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:23:48.504 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:48.504 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:48.513 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:48.549 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:48.563+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:48.637 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:48.638 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:48.646 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:48.647 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:48.649 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:48.663+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:48.667 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:48.687 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:48.688 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:23:48.688 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:23:48.688 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:23:48.750 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:48.763+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:48.774 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:48.774 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:48.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:48.774 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.278626312, 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.775 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:48.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:48.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:48.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:48.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:48.785 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:48.786 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.278514104999999, 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:48.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:48.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:48.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:48.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:48.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.797 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.278423867000001, 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.278307223999999, 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:48.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.819 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.278158855000001, 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:48.820 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:48.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:48.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:48.831 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.277969486, 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:48.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:48.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:48.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.842 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.277755564, 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:48.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:48.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:48.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:48.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:48.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.277556662999999, 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.854 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:48.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:48.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.277375425999999, 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.864 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:48.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:48.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:48.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.276748112, 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:48.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:48.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:48.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.883 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.276450769, 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:48.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:48.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:48.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.276043956999999, 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.275944739, 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.275749773000001, 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.275188701000001, 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.275030220000001, 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.274897309, 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:48.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:48.943 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.274728700000001, 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:48.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.039024939000001, 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:23:48.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:23:48.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:23:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:23:48.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.961 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.038897911999999, 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:48.962 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:23:48.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:23:48.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:23:48.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:48.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:23:48.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:48.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.972 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:48.975 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:23:48.975 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.1 primary-affinity to 0 (802) 2020-11-16T22:23:48.976 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:48.976 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:48.988+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:49.075 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:49.089+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:49.113 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:49.113 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:49.113 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:49.114 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:49.122 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:49.176 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:49.190+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:49.246 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:49.247 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:49.247 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:23:49.247 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:23:49.247 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:49.255 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:49.299 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:49.312+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:49.378 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:49.392+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.381 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 39.305120158000001, 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 39.193690476999997, 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.390 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:23:49.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:23:49.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:23:49.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.885253926000001, 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.398 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:23:49.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:23:49.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:23:49.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:23:49.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.885138626, 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:23:49.408 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:23:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:23:49.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.417 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.885029866, 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:23:49.418 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.427 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.884947517000001, 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:23:49.428 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:23:49.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:23:49.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:23:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:23:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.884859709000001, 2020-11-16T22:23:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:49.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:23:49.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:23:49.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.447 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.884765170000001, 2020-11-16T22:23:49.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:23:49.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:23:49.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:23:49.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:23:49.457 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.884678984000001, 2020-11-16T22:23:49.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:49.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:23:49.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:23:49.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:23:49.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:23:49.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.884578599000001, 2020-11-16T22:23:49.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:23:49.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:23:49.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:23:49.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.879465278000001, 2020-11-16T22:23:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:23:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:23:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:23:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.879359104999999, 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:23:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:23:49.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:23:49.493 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:49.494 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.879185095, 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:49.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:23:49.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:23:49.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:23:49.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:49.502 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.879088815999999, 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:23:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:23:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:23:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.878901114000001, 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.87870916, 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:23:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.878546596, 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:23:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:23:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.878370734000001, 2020-11-16T22:23:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:23:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:23:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:23:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.878202688999998, 2020-11-16T22:23:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:23:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:23:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:23:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.877961558999999, 2020-11-16T22:23:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:49.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:49.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:23:49.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:23:49.556 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.557 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:23:49.557 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:23:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:49.558 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:49.559 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:49.572+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:49.659 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:49.673+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:49.694 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:49.694 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:49.694 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:23:49.694 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:49.703 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:49.760 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:49.774+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:49.827 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:23:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:23:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:23:49.828 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:49.836 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:49.861 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:49.875+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:49.962 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:49.976+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:50.004 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:50.004 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:23:50.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:23:50.004 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:23:50.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.772492026000002, 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:50.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:23:50.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:23:50.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:23:50.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.928894687000003, 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:50.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:50.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:23:50.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:23:50.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:23:50.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:23:50.023 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.817430835000003, 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:23:50.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:23:50.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:23:50.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:23:50.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:23:50.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.034 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.508562509999997, 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:23:50.035 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:23:50.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:50.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.044 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.508384771999999, 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:23:50.045 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.054 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.508204653, 2020-11-16T22:23:50.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:23:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:23:50.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:23:50.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:23:50.064 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.508042392, 2020-11-16T22:23:50.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:50.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:23:50.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:23:50.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:23:50.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:23:50.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507911155000002, 2020-11-16T22:23:50.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:23:50.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:23:50.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:23:50.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:50.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507739227999998, 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:23:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:23:50.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:50.095 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507545223999998, 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.105 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507344363999998, 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:23:50.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:23:50.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:23:50.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:23:50.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.115 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507176686999998, 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:23:50.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:23:50.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:23:50.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:23:50.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.507019350999997, 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:23:50.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:23:50.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:23:50.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:50.137 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.506889381999997, 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:50.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:23:50.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.503684438999997, 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:23:50.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.503571282000003, 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:23:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:23:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.503511992, 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.503133118000001, 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.169 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:23:50.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.50291361, 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:23:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:23:50.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.502870432000002, 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:50.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:23:50.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:50.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:23:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:50.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:50.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:50.191 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:50.193 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:50.206+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:50.193 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:50.293 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:50.307+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:50.329 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:50.330 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:50.330 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:50.330 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:50.339 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:50.395 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:50.409+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:50.496 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:50.509+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:50.504 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:50.505 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:50.505 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:23:50.505 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:23:50.505 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:50.514 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:50.597 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:50.610+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:50.681 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:50.681 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:23:50.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.185579625000003, 2020-11-16T22:23:50.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:23:50.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:23:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:23:50.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:23:50.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.185467418000002, 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.694 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:50.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:23:50.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:23:50.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.704 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.185377180000003, 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:23:50.705 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:23:50.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:23:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:23:50.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:23:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.185260536999998, 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:23:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:23:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:23:50.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:23:50.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:23:50.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.185112168000003, 2020-11-16T22:23:50.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:50.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:23:50.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:23:50.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:23:50.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:23:50.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.184922798999999, 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.740 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:23:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:23:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:23:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:23:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:23:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.184708876999998, 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:23:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:50.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.184509976000001, 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:23:50.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:23:50.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.184328739000001, 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.773 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:23:50.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:23:50.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:23:50.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:23:50.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.183701425000002, 2020-11-16T22:23:50.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:23:50.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:23:50.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:23:50.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:23:50.792 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:50.793 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.183404082000003, 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:50.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:23:50.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:23:50.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.803 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.182997270000001, 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:23:50.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:23:50.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:23:50.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:23:50.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.811 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.182898051999999, 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:23:50.812 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.821 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.182703085999997, 2020-11-16T22:23:50.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:23:50.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:23:50.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:23:50.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:23:50.831 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.182142014, 2020-11-16T22:23:50.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:23:50.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:23:50.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.839 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.181983533, 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:23:50.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.846 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.181850621999999, 2020-11-16T22:23:50.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:23:50.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:23:50.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.181682013, 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:23:50.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:23:50.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:23:50.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:23:50.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 13.945978252, 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.863 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:23:50.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:23:50.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:23:50.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:23:50.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:23:50.873 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:50.874 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 13.945851225, 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:23:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:23:50.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:23:50.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:23:50.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:23:50.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:23:50.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:23:50.885 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:50.886 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:50.888 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:50.901+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:50.888 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:50.976 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:23:50.987 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:51.001+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:51.025 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:51.025 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:23:51.025 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:23:51.026 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:51.034 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:51.042 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:51.063 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:51.083 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:51.084 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:23:51.084 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:23:51.085 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:23:51.087 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:51.102+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:51.158 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:51.159 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:51.167 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:51.188 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:51.202+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:51.289 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:51.303+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:51.291 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:51.291 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:51.300 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:51.389 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:51.403+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:51.436 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:51.436 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:51.445 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:51.491 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:51.504+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:51.592 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:51.605+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:51.795 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:51.795 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:51.796 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:51.796 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:51.809+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:51.867 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:23:51.896 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:51.910+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:51.921 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:51.921 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:51.930 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:51.997 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:52.011+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:52.066 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.066 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.076 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:52.118 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:52.132+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:52.199 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:52.212+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:52.201 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.201 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.210 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:23:52.300 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:52.313+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:52.375 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.375 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.384 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:23:52.428 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:23:52.441+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:23:52.502 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:52.516+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:52.521 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.521 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.530 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:23:52.602 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:52.616+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:52.694 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.695 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.703 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:52.717+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:52.704 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:23:52.803 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:52.817+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:52.828 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.828 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.837 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:23:52.904 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:52.919+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:52.973 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:52.973 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:52.982 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:23:53.023 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:53.037+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:53.095 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:23:53.106 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:53.120+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:53.147 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:23:53.147 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:23:53.156 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:23:53.207 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:23:53.221+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:23:53.247 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:53.266 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:23:53.286 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:23:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:23:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:23:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:23:53.307 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:53.321+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:53.322 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:23:53.323 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.091148799000003, 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:53.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:53.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:23:53.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:53.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:23:53.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:53.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:23:53.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:23:53.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:23:53.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:23:53.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:23:53.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:53.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.247551459999997, 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:23:53.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:23:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.539 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:53.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:53.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:53.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:53.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:53.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:23:53.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:53.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:23:53.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:23:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:53.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:23:53.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:23:53.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:23:53.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:53.953 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.136087607999997, 2020-11-16T22:23:53.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:23:53.955 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:53.955 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:53.955 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:23:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:23:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:23:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:23:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:23:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:54.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:23:54.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:23:54.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:54.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:23:54.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:23:54.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:54.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:54.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:54.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.574 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:54.574 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.827219282999998, 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:23:54.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:54.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:23:54.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:23:54.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:54.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:23:54.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:23:54.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:23:54.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:54.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:23:54.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:23:54.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:54.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:23:54.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:23:54.990 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:54.990 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:54.990 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:54.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.827041545, 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:23:54.991 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:54.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:23:54.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:23:54.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:54.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:54.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:54.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:23:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:23:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:23:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:23:55.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:23:55.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:55.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826861426000001, 2020-11-16T22:23:55.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:23:55.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:55.409 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:55.409 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:55.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:55.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:23:55.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:55.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:23:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:23:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:23:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:23:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:23:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:23:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:23:55.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:23:55.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:55.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826699165000001, 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:23:55.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:23:55.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:55.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:55.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:55.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:23:55.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:23:56.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:56.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:23:56.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:23:56.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:56.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:23:56.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:23:56.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:56.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:23:56.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:56.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826567928000003, 2020-11-16T22:23:56.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:56.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:56.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:56.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:23:56.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:23:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:23:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:23:56.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:56.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:23:56.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:23:56.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:56.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:23:56.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826396000999999, 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:56.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:56.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:56.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:56.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:23:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:23:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:23:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:23:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:57.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:23:57.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:23:57.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:23:57.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826201996999998, 2020-11-16T22:23:57.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:23:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:23:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:23:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:23:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:23:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:23:57.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:57.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:57.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:57.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.826001136999999, 2020-11-16T22:23:57.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:23:57.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:57.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:57.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:23:57.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:23:57.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:57.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:23:57.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:23:57.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:23:57.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:57.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:57.907 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:58.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:58.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.825833459999998, 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:58.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:23:58.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:23:58.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:23:58.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:58.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:23:58.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:23:58.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:58.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:23:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:23:58.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:23:58.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:58.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:23:58.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:23:58.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:58.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:58.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.825676123999997, 2020-11-16T22:23:58.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:58.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:58.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:23:58.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:58.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:23:58.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:58.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.825546154999998, 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:23:58.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:58.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:58.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:23:58.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:58.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:23:58.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:58.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:23:59.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:23:59.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:23:59.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.822341211999998, 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:23:59.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:23:59.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:23:59.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:23:59.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.363 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:23:59.363 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:59.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.822228054999997, 2020-11-16T22:23:59.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:23:59.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.566 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:59.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:23:59.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:23:59.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.573 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.822168765000001, 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:23:59.574 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:23:59.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:23:59.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:23:59.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:59.582 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.821789891000002, 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:23:59.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.821570383000001, 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:23:59.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:23:59.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:23:59.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:59.597 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.821527205000002, 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:23:59.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:23:59.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:23:59.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:23:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:23:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:23:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:23:59.806 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:23:59.806 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:23:59.807 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:23:59.808 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:23:59.808 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:23:59.808 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:23:59.808 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:23:59.809 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:23:59.809 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:23:59.810 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:23:59.811 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:59.822+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:59.812 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:23:59.908 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:23:59.923+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:23:59.946 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:23:59.946 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:23:59.947 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:23:59.950 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:23:59.950 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:23:59.950 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:23:59.951 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:23:59.960 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:23:59.969 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:23:59.989 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:00.009 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:24:00.010 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:00.023+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:00.010 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:24:00.085 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.085 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.094 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:00.109 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:00.123+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:00.211 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:00.225+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:00.219 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.220 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.228 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:00.312 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:00.326+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:00.356 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.356 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.365 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:00.413 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:00.426+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:00.514 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:00.527+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:00.529 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.530 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.539 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:00.615 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:00.629+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:00.663 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.663 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.672 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:24:00.716 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:00.730+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:00.807 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.808 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.817 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:24:00.817 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:00.830+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:00.883 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:24:00.918 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:00.931+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:00.942 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:00.942 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:00.951 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:24:01.019 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.032+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.075 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.075 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.084 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:01.127 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.140+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.215 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.215 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.220 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:01.234+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:01.224 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:01.321 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:01.335+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:01.388 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.389 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.398 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:01.441 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.454+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.522 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.523 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.523 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.535+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.532 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:01.624 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.637+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.666 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.667 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.676 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:01.724 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:01.738+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:01.801 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.801 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.811 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:01.825 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:01.838+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:01.925 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:01.940+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:01.974 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:01.974 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:01.983 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:24:02.022 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:02.026 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:02.040+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:02.125 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:02.126 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:02.128 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:02.142+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:02.135 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:24:02.174 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:02.194 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:02.213 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:24:02.214 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:02.215 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:02.215 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:24:02.228 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:02.242+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:02.262 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:24:02.262 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:24:02.262 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:24:02.262 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:24:02.262 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:24:02.271 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:24:02.329 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:02.342+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:02.396 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:24:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.164933312000002, 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:24:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:24:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 52.321335972999996, 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:24:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:24:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:24:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 52.209872120999997, 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:24:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:24:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:24:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:24:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:24:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.901003795999998, 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:24:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:24:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:24:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:24:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.900826058, 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:24:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:24:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.900645939, 2020-11-16T22:24:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:24:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:24:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:24:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.900483678000001, 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:24:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:24:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.900352441000003, 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:24:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:24:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:24:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.900180513999999, 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:24:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:24:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:24:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:24:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.899986509999998, 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.899785649999998, 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:24:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:24:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:24:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.899617972999998, 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:24:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:24:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:24:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:24:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.899460636999997, 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:24:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:24:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.899330667999998, 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:24:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:24:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.896125724999997, 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:24:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:24:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:24:02.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.896012568000003, 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:24:02.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:24:02.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:24:02.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.895953278, 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:24:02.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:24:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:24:02.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:24:02.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:02.562 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.895574404000001, 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:24:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:24:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.895354896000001, 2020-11-16T22:24:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:24:02.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:24:02.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.895311718000002, 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:24:02.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:24:02.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:24:02.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:02.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:02.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:02.587 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:24:02.588 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:02.601+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:02.588 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:02.689 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:02.703+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:02.725 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:02.726 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:02.726 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:02.726 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:02.736 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:02.790 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:02.804+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:02.869 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:02.896 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:24:02.912 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:02.925+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:02.991 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:03.004+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:02.996 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:02.996 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.50108814, 2020-11-16T22:24:02.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:02.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:02.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:03.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:03.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:03.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:03.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500975932999999, 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:03.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:03.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.019 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500885695000001, 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:03.020 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:03.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:03.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:03.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:03.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.031 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500769052000003, 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:03.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:03.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500620683000001, 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:03.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:03.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:03.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.053 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500431313999997, 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:03.054 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500217392000003, 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.500018490999999, 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.499837253999999, 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:03.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:03.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:03.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:03.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.49920994, 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.097 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:03.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:03.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.107 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.498912597, 2020-11-16T22:24:03.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:03.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:03.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:03.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:03.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.118 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.498505784999999, 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:03.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:03.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:03.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.498406567000004, 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:03.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:03.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:03.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.498211601000001, 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:03.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:03.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.497650528999998, 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:03.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:03.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.153 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.497492047999998, 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:03.154 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.161 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.497359136999997, 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:03.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:03.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:03.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.497190527999997, 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.169 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:03.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:03.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.176 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.261486767000001, 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:03.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:03.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:03.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:03.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:03.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.26135974, 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:03.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:03.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:03.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:03.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:03.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.199 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:03.199 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:03.199 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:03.200 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:03.201 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:03.214+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:03.301 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:03.315+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:03.337 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:03.338 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:03.347 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:03.402 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:03.416+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:03.503 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:03.517+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:03.511 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:03.511 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:03.512 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:03.512 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:03.512 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:03.521 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:03.605 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:03.619+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:03.687 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:03.687 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:03.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:03.687 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 53.611263688000001, 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.688 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:03.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:03.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:03.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.695 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 53.499834006999997, 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:03.696 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.191397455999997, 2020-11-16T22:24:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.191282156, 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.191173396000003, 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.191091047, 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.191003238999997, 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.190908700000001, 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.190822513999997, 2020-11-16T22:24:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.190722129000001, 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.185608807999998, 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.185502634999999, 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.185328624999997, 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.185232345999999, 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.185044644000001, 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.18485269, 2020-11-16T22:24:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.184690126, 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.184514264000001, 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.184346218999998, 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.184105088999999, 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:03.870 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:03.871 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:03.884+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:03.871 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:24:03.897 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:24:03.972 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:03.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:04.007 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:24:04.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:24:04.008 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:24:04.008 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:24:04.017 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:24:04.073 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:04.086+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:04.141 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:24:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:24:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:24:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:24:04.142 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:24:04.150 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:24:04.173 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:04.187+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:04.225 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:04.274 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:04.288+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:04.275 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:1 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e8)", 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:04.276 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.044011746000002, 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995, 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:04.277 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244566+0000", 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1554999999999999e-05 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244598+0000", 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.755e-06 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244603+0000", 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2020-11-16T22:24:04.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244616+0000", 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2519999999999999e-06 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244624+0000", 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.64e-05 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244701+0000", 2020-11-16T22:24:04.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.789e-05 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:09.244719+0000", 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84261598599999998 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087335+0000", 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164802 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.087499+0000", 2020-11-16T22:24:04.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.84293338399999995 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:2 1.0 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:04.285 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.200414406999997, 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998, 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:04.286 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088163+0000", 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.841e-05 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088202+0000", 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7660000000000004e-06 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088208+0000", 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1681e-05 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088219+0000", 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9820000000000002e-06 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088226+0000", 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2923999999999999e-05 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088259+0000", 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109269 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088368+0000", 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012339600000000001 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:04.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.088492+0000", 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061693999999999996 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.089109+0000", 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075774099999999997 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096686+0000", 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4972999999999997e-05 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096741+0000", 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0207000000000002e-05 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.096791+0000", 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086279579999999998 2020-11-16T22:24:04.295 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4166.0:4 1.0 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e9)", 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.088950554999997, 2020-11-16T22:24:04.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997, 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4166", 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.138:0/1284830027", 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 4 2020-11-16T22:24:04.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199627+0000", 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6470999999999997e-05 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199664+0000", 2020-11-16T22:24:04.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9009999999999999e-06 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199670+0000", 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3867e-05 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199684+0000", 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.711e-06 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199689+0000", 2020-11-16T22:24:04.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8494e-05 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199738+0000", 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117642 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199855+0000", 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371100000000001 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.199989+0000", 2020-11-16T22:24:04.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012998420000000001 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.201289+0000", 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031390300000000001 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204428+0000", 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1907999999999998e-05 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204480+0000", 2020-11-16T22:24:04.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.413e-05 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:10.204514+0000", 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048867069999999997 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.306 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:13 3.5 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.780082229999998, 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004, 2020-11-16T22:24:04.307 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508496+0000", 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2387999999999999e-05 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508568+0000", 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0939e-05 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508579+0000", 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4161000000000001e-05 2020-11-16T22:24:04.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508603+0000", 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5599999999999994e-06 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508612+0000", 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8668999999999997e-05 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508670+0000", 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000167171 2020-11-16T22:24:04.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508838+0000", 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016276299999999999 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509000+0000", 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039887890000000004 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512989+0000", 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5161999999999995e-05 2020-11-16T22:24:04.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513074+0000", 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1797000000000004e-05 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8681999999999998e-05 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513205+0000", 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047090810000000004 2020-11-16T22:24:04.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:14 3.5 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779904492, 2020-11-16T22:24:04.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997, 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:04.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508673+0000", 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6181999999999998e-05 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508740+0000", 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0628999999999999e-05 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508750+0000", 2020-11-16T22:24:04.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1546999999999999e-05 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508772+0000", 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7559999999999996e-06 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508777+0000", 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056667399999999998 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509344+0000", 2020-11-16T22:24:04.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012436500000000001 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509469+0000", 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000249996 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509719+0000", 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003306238 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513025+0000", 2020-11-16T22:24:04.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015728739999999999 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514598+0000", 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2305999999999994e-05 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514680+0000", 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878999999999994e-05 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514747+0000", 2020-11-16T22:24:04.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060734459999999997 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:15 3.5 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779724373000001, 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002, 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508853+0000", 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1419000000000002e-05 2020-11-16T22:24:04.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508925+0000", 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9177e-05 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508955+0000", 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:24:04.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.508963+0000", 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010266940000000001 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509989+0000", 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017857999999999999 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510168+0000", 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022891599999999999 2020-11-16T22:24:04.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510397+0000", 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028227320000000001 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513220+0000", 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001619625 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7512000000000002e-05 2020-11-16T22:24:04.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514887+0000", 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8424999999999999e-05 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514925+0000", 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060716900000000002 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.337 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:16 3.5 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779562112000001, 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001, 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509016+0000", 2020-11-16T22:24:04.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509078+0000", 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8250000000000001e-06 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509084+0000", 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-05 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509095+0000", 2020-11-16T22:24:04.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6540000000000002e-06 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509102+0000", 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650629 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510752+0000", 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510882+0000", 2020-11-16T22:24:04.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033157100000000001 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030566590000000002 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514270+0000", 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070631100000000005 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514977+0000", 2020-11-16T22:24:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4818000000000001e-05 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515051+0000", 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9466000000000003e-05 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.515111+0000", 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060950570000000001 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30 3.3 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779430875000003, 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004, 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509147+0000", 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3491000000000005e-05 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509220+0000", 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6419999999999996e-06 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509229+0000", 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4871e-05 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509244+0000", 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509250+0000", 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4571999999999999e-05 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509295+0000", 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013799999999999999 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509433+0000", 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168643 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509601+0000", 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026614189999999999 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512263+0000", 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001852722 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514115+0000", 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9374000000000004e-05 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514185+0000", 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123478 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051611950000000004 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:26 3.3 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779258947999999, 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999, 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.358 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509319+0000", 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6636000000000001e-05 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509416+0000", 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064000000000001e-05 2020-11-16T22:24:04.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509426+0000", 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7567e-05 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509443+0000", 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0419999999999997e-06 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509450+0000", 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050681100000000002 2020-11-16T22:24:04.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014567200000000001 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510103+0000", 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022106400000000001 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510324+0000", 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029061550000000001 2020-11-16T22:24:04.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513230+0000", 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030300230000000002 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516260+0000", 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101000000000001e-05 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516312+0000", 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4974e-05 2020-11-16T22:24:04.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516357+0000", 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070381089999999999 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27 3.3 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.779064943999998, 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001, 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509513+0000", 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4764000000000004e-05 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509598+0000", 2020-11-16T22:24:04.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9748000000000001e-05 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509617+0000", 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5024999999999999e-05 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509632+0000", 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6170000000000003e-06 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509639+0000", 2020-11-16T22:24:04.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011325350000000001 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510772+0000", 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027255399999999999 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511044+0000", 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198844 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511243+0000", 2020-11-16T22:24:04.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019955929999999999 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513239+0000", 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031693939999999999 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516408+0000", 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2794999999999999e-05 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516451+0000", 2020-11-16T22:24:04.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3170000000000003e-05 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516484+0000", 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069710390000000001 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:28 3.5 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.778864083999999, 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964, 2020-11-16T22:24:04.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509714+0000", 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5376999999999993e-05 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509789+0000", 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8529999999999998e-06 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509798+0000", 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4803999999999999e-05 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509813+0000", 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509821+0000", 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001775118 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511596+0000", 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000231886 2020-11-16T22:24:04.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511828+0000", 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042716999999999999 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020281140000000001 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514283+0000", 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026190440000000001 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516902+0000", 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8769e-05 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516951+0000", 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3676e-05 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516995+0000", 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007280964 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.387 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:29 3.5 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.778696406999998, 2020-11-16T22:24:04.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999, 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509881+0000", 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8473000000000007e-05 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509950+0000", 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9609999999999999e-06 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509957+0000", 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6022999999999999e-05 2020-11-16T22:24:04.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509973+0000", 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.509979+0000", 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032745980000000001 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513254+0000", 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014960599999999999 2020-11-16T22:24:04.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513403+0000", 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018982499999999999 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513593+0000", 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034535809999999998 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014794499999999999 2020-11-16T22:24:04.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517195+0000", 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5833000000000005e-05 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517260+0000", 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1400000000000003e-05 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517312+0000", 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074303659999999999 2020-11-16T22:24:04.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32 3.3 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:04.398 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.778539070999997, 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999, 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:04.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510039+0000", 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3769e-05 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1660000000000002e-06 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510098+0000", 2020-11-16T22:24:04.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3750000000000001e-05 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510121+0000", 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510126+0000", 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001461078 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511588+0000", 2020-11-16T22:24:04.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012042199999999999 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.511708+0000", 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049628429999999998 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516671+0000", 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066320509999999999 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:33 3.3 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.778409101999998, 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997, 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:24:04.406 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3548999999999998e-05 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510232+0000", 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1739999999999999e-06 2020-11-16T22:24:04.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510237+0000", 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3122999999999999e-05 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510261+0000", 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6100000000000002e-06 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.510264+0000", 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020729189999999999 2020-11-16T22:24:04.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512337+0000", 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214195 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.512551+0000", 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004378949 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516930+0000", 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067615189999999997 2020-11-16T22:24:04.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:9 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.775204158999998, 2020-11-16T22:24:04.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001, 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513374+0000", 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3702000000000006e-05 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513457+0000", 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6579999999999997e-06 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513467+0000", 2020-11-16T22:24:04.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3898e-05 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513481+0000", 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5250000000000002e-06 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8056999999999997e-05 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513536+0000", 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6577000000000001e-05 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513552+0000", 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031543119999999998 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516706+0000", 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5639e-05 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516762+0000", 2020-11-16T22:24:04.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033883680000000001 2020-11-16T22:24:04.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.422 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:22 3.6 e16/14 3:717a0223:::608.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.775091002000003, 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002, 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513487+0000", 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021739700000000001 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513704+0000", 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513714+0000", 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2106000000000001e-05 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513726+0000", 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513733+0000", 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5708000000000002e-05 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513799+0000", 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0210999999999998e-05 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513839+0000", 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002896577 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516736+0000", 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9528000000000003e-05 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.516795+0000", 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033082290000000002 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:10 3.0 e16/14 3:0bd6d154:::602.00000000:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.775031712000001, 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001, 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.431 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513546+0000", 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144938 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513691+0000", 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6038999999999999e-05 2020-11-16T22:24:04.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513717+0000", 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4112e-05 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513731+0000", 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3767000000000001e-05 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513755+0000", 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027623600000000002 2020-11-16T22:24:04.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514031+0000", 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1039e-05 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514052+0000", 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031639060000000002 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517216+0000", 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0521999999999999e-05 2020-11-16T22:24:04.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517297+0000", 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037505590000000001 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:11 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'7, mlcod=0'0)", 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.774652838000002, 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164, 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.513925+0000", 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173654 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514099+0000", 2020-11-16T22:24:04.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1758e-05 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514110+0000", 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5364000000000001e-05 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514126+0000", 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514134+0000", 2020-11-16T22:24:04.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015859399999999999 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514293+0000", 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6288999999999999e-05 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002983472 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517303+0000", 2020-11-16T22:24:04.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3521000000000004e-05 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517366+0000", 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003441164 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.6 epoch 16/14 rep_tid 9 entries 16'3 (0'0) error 3:7bf7e7b5:::200.00000006:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.774433330000001, 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002, 2020-11-16T22:24:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514145+0000", 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1866000000000001e-05 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514196+0000", 2020-11-16T22:24:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514209+0000", 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0058999999999999e-05 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514230+0000", 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0208e-05 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514240+0000", 2020-11-16T22:24:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012394199999999999 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514364+0000", 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002929152 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517293+0000", 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031482950000000002 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:12 3.0 e16/14 3:0d89b25e:::603.00000000:head v 16'8, mlcod=0'0)", 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.774390152000002, 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002, 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:24:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514188+0000", 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0927999999999994e-05 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514279+0000", 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:24:04.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514285+0000", 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.605e-05 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:24:04.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514301+0000", 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8950000000000001e-06 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514308+0000", 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025387899999999999 2020-11-16T22:24:04.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514562+0000", 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5758999999999999e-05 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:24:04.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.514588+0000", 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027818389999999999 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517370+0000", 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7870000000000002e-05 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:24:04.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:23:17.517408+0000", 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032198600000000002 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:24:04.462 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:24:04.462 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:24:04.463 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:04.464 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:04.464 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:24:04.464 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:04.464 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:04.464 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:24:04.465 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:04.477+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:04.465 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:04.564 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:04.578+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:04.602 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:04.603 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:04.612 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:04.665 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:24:04.679+0000 7f721d186700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20832) UID: 0 2020-11-16T22:24:04.735 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:04.735 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:04.744 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:04.787 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:04.801+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:04.866 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:04.880+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:04.869 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:04.870 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:04.878 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:04.956 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:24:04.956 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:24:04.956 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2] 2020-11-16T22:24:04.957 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.2 doing powercycle of ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:24:04.957 INFO:teuthology.orchestra.console:Power off smithi122 2020-11-16T22:24:04.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:24:04.970 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:04.983+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:04.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:24:05.015 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.015 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.024 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:05.107 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.121+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.171 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.185+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.190 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.190 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.199 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:05.282 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.296+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.363 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.363 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.372 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:05.373 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:05.386+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:05.474 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.488+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.501 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.502 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.511 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:05.575 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.589+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.635 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.635 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.644 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:05.687 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:05.701+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:05.769 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.769 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.775 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:05.789+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:05.778 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:05.876 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:05.890+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:05.905 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:05.905 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:05.909 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:24:05.914 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:05.977 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:05.991+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:06.037 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:06.037 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:06.046 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:06.078 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:06.091+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:06.169 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:06.170 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:06.179 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:06.193+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:06.180 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:06.279 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:06.293+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:06.306 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:06.306 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:06.315 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:06.380 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:06.394+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:06.439 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:06.440 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:06.449 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:06.466 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:06.489 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:06.503+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:06.583 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:06.597+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:06.606 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:06.615 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:06.615 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:06.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119852801999997, 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:06.616 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:06.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:06.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:06.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:06.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:06.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:06.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:06.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:06.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:06.627 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119740595000003, 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:06.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:06.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:06.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:06.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:06.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:06.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:06.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119650356999998, 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:06.639 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:06.842 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:06.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:06.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:06.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:06.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:06.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:06.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:06.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:06.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:06.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:07.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.055 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119533713999999, 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:07.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:07.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:07.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:07.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:07.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:07.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:07.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:07.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:07.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.269 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119385344999998, 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:07.270 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:07.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:07.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:07.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:07.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:07.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:07.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:07.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:07.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:07.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:07.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:07.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.686 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.119195976, 2020-11-16T22:24:07.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:07.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:07.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:07.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:07.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:07.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:07.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:07.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:07.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:07.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:07.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:08.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:08.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:08.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.118982054, 2020-11-16T22:24:08.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:08.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:08.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:08.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:08.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:08.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:08.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:08.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:08.317 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:08.318 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.118783153000003, 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:08.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:08.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:08.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:08.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:08.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.328 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.118601916000003, 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:08.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:08.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:08.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:08.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.117974601999997, 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.339 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:08.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:08.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.117677258999997, 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.349 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:08.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:08.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:08.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:08.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.117270447000003, 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.560 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:08.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:08.567 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.117171229, 2020-11-16T22:24:08.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:08.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:08.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:08.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:08.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:08.578 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.116976262999998, 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:08.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:08.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:08.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.588 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.116415191000002, 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:08.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:08.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:08.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.116256710000002, 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.596 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:08.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:08.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.603 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.116123799, 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:08.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:08.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:08.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 49.115955190000001, 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.611 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:08.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:08.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:08.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:08.618 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 29.880251429000001, 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:08.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:08.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 29.880124402, 2020-11-16T22:24:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:08.834 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:08.834 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:08.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:08.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:08.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:08.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:08.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:08.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.844 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.845 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.845 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:08.845 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:08.845 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:08.848 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:08.848 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:24:08.848 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:08.848 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:08.848 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:24:08.849 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:24:08.849 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:08.850 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:08.860+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:08.947 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:08.961+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:08.988 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:08.989 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:08.998 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:09.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:24:09.048 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:09.062+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:09.124 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:09.125 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:09.134 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:09.174 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:09.188+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:09.250 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:09.263+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:09.299 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 59.223089017, 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 59.111659336000002, 2020-11-16T22:24:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:09.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:09.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:09.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.803222785000003, 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:09.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:09.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:09.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:09.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.326 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.803107484999998, 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:09.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:09.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.336 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802998725000002, 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:09.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:09.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:09.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.346 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802916375999999, 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:09.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:09.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:09.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:09.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.356 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802828568000002, 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:09.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:09.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:09.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:09.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.366 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802734029, 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:09.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:09.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:09.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:09.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802647843000003, 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:09.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:09.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.802547457999999, 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:09.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:09.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.797434137000003, 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.797327963999997, 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.405 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:09.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:09.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.797153954000002, 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.413 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:09.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:09.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:09.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.797057674999998, 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.421 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:09.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:09.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:09.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.796869973, 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.429 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:09.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:09.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.796678018999998, 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:09.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:09.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:09.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.796515454999998, 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:09.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:09.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:09.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.796339592999999, 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:09.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:09.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:09.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.796171547999997, 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 51.795930417999998, 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:09.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:09.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:09.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:09.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:09.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:09.475 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:09.476 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:09.477 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:09.490+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:09.477 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:09.578 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:09.591+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:09.614 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:09.615 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:09.624 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:09.679 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:09.692+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:09.748 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:09.749 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:09.749 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:09.749 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:09.749 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:09.757 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:09.798 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:09.812+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:09.880 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:09.894+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:09.926 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.430637220999998, 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:09.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:09.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:09.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:10.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:10.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:10.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:10.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:10.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:10.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:10.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:10.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.343 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.430525013999997, 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:10.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:10.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:10.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:10.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:10.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:10.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:10.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:10.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:10.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:10.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:10.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:10.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:10.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:10.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:10.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:10.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:10.761 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.430434775999998, 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:10.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:10.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:10.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:10.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:10.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:10.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:10.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:10.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:10.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:10.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:10.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:10.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:11.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:11.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:11.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:11.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:11.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:11.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:11.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:11.179 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:11.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:11.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.382 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:11.382 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:11.382 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.430318133, 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:11.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:11.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:11.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:11.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:11.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:11.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:11.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:11.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:11.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:11.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:11.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:11.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:11.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:11.799 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:11.800 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.430169763999999, 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:11.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:11.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:11.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:11.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:11.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:12.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:12.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:12.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:12.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:12.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:12.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:12.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:12.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:12.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:12.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:12.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.429980395000001, 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:12.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:12.422 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:12.422 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:12.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:12.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:12.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:12.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:12.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:12.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:12.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:12.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:12.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:12.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:12.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:12.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:12.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:12.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:12.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:12.839 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.429766473000001, 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:12.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:12.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:12.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:12.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:12.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:13.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:13.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:13.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:13.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:13.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:13.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:13.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:13.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:13.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.429567572000003, 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:13.262 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:13.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:13.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:13.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:13.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:13.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:13.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:13.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:13.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:13.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:13.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:13.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:13.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:13.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:13.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.429386334999997, 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:13.678 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:13.679 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:13.679 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:13.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:13.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:13.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:13.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:13.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:13.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:13.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:13.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:14.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:14.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:14.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:14.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.095 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:14.095 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:14.095 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.428759020999998, 2020-11-16T22:24:14.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:14.298 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:14.298 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:14.298 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:14.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:14.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.714 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:14.714 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:14.714 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.428461677999998, 2020-11-16T22:24:14.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:14.714 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:14.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:14.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:14.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:14.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:14.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:14.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:14.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:14.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:14.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:14.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:14.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:14.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:14.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:14.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:14.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:14.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:14.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:14.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:14.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:14.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:14.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:14.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:14.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:15.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:15.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.134 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:15.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.428054865999997, 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:15.135 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:15.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:15.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:15.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:15.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:15.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:15.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:15.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:15.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.427955648000001, 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:15.347 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:15.550 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:15.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:15.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:15.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:15.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:15.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:15.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:15.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:15.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:15.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:15.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.762 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:15.763 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:15.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.763 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:15.763 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:15.966 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.427760681999999, 2020-11-16T22:24:15.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:15.966 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:15.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:15.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:15.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:15.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:15.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:15.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:15.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:16.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:16.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:16.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:16.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:16.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:16.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:16.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:16.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:16.179 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:16.179 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.427199610000002, 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:16.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:16.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:16.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:16.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:16.592 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:16.593 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.427041129000003, 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:16.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:16.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:16.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:16.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:16.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:16.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:16.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:16.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:16.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:16.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:16.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:16.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:17.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:17.006 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.006 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.006 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.426908218000001, 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:17.007 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:17.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:17.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:17.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:17.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:17.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.426739609000002, 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:17.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:17.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.191035847999999, 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:17.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:17.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:17.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:17.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:17.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.643 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.190908821000001, 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:17.644 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:17.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:17.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:17.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:17.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:17.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:17.655 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.656 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:17.656 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:17.656 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:17.657 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:17.658 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:24:17.658 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:24:17.669 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:17.671+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:17.670 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:17.758 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:17.772+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:17.808 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:17.808 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:17.808 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:17.808 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:17.817 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:17.835 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:17.859 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:17.859 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:17.859 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:17.859 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:17.860 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:17.860 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:17.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:17.861 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:17.875+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:17.870 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2020-11-16T22:24:17.962 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:17.975+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:17.985 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:17.985 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:17.994 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:18.044 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:24:18.077 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.091+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:18.160 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.160 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.163 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:18.176+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:18.169 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:18.263 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:18.277+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:18.307 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.307 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.316 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:18.364 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.379+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:18.465 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.479+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:18.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2020-11-16T22:24:18.481 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.481 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.490 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:18.566 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:18.580+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:18.615 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.615 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.624 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:18.667 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.681+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:18.751 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.751 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.760 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:18.768 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:18.782+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:18.869 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.883+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:18.924 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:18.924 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:18.933 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:18.974 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:18.988+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.070 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:19.084+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.098 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.098 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.107 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:19.172 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:19.185+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.236 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.236 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.245 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:19.288 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:19.302+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:19.370 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.370 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.373 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:19.388+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.379 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:19.474 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:19.487+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:19.545 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.545 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.554 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:19.598 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:19.612+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.675 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:19.689+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:19.683 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.683 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.692 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:19.776 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:19.790+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:19.856 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.856 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.862 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:19.865 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:19.908 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:19.922+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:19.977 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:19.991+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:19.989 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:19.989 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:19.998 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:20.013 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:20.033 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:20.033 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:20.036 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:20.036 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:20.055 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:24:20.078 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:20.092+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:20.125 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:20.125 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:20.125 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:20.126 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:20.134 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:20.217 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:20.231+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:20.280 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:20.293+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:20.300 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:20.301 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:20.309 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:20.380 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:20.395+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:20.476 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:20.476 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:20.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.980519176999998, 2020-11-16T22:24:20.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:20.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:20.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:20.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:20.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:20.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:20.489 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.980406969999997, 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:20.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:20.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:20.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:20.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:20.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:20.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.501 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.980316731999999, 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:20.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:20.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:20.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:20.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.513 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.980200089, 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:20.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:20.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:20.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:20.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.525 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.980051719999999, 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.979862351000001, 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.538 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:20.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.979648429000001, 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.979449528000004, 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.979268290999997, 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.978640976999998, 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:20.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:20.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.594 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.978343633999998, 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:20.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:20.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:20.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:20.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.604 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.977936821999997, 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:20.605 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:20.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:20.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.613 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.977837604000001, 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:20.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:20.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:20.623 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:20.624 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.977642637999999, 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:20.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:20.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:20.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:20.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.977081566000003, 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.635 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:20.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:20.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.976923085000003, 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:20.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:20.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:20.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:20.650 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.976790174000001, 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:20.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:20.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:20.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.657 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.976621565000002, 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:20.658 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:20.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:20.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 43.740917803999999, 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.666 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:20.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:20.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:20.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:20.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 43.740790777000001, 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:20.678 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:20.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:20.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:20.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:20.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:20.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:20.689 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:20.690 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:20.691 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:20.705+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:20.691 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:20.792 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:20.806+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:20.820 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:20.821 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:20.821 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:20.821 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:20.830 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:20.892 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:20.906+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:20.993 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:21.007+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:20.995 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:20.996 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:21.004 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:21.059 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:24:21.093 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:21.107+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:21.131 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:21.131 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:21.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:21.131 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:21.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 71.054883008000004, 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.132 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:21.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:21.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.943453327, 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.141 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:21.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:21.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:21.148 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.149 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.635016776000001, 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:21.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:21.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:21.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:21.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:21.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:21.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:21.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:21.160 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.160 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.160 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.161 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.161 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:21.161 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634901476000003, 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:21.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:21.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:21.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:21.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:21.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.172 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634792716, 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:21.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:21.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:21.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:21.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.182 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634710366999997, 2020-11-16T22:24:21.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:21.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:21.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:21.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:21.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.193 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634622559, 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:21.194 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:21.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:21.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634528019999998, 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:21.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:21.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:21.214 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:21.215 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634441834, 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:21.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:21.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:21.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.225 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.634341448999997, 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:21.226 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:21.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:21.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:21.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:21.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:21.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:21.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:21.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:21.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:21.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:21.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.237 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.629228128000001, 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:21.238 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:21.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:21.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:21.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.629121955000002, 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.247 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:21.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:21.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:21.254 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:21.255 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628947945, 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:21.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:21.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628851666000003, 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628663963999998, 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628472010000003, 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628309446000003, 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:21.495 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.628133583999997, 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:21.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:21.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:21.503 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.627965539000002, 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:21.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:21.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:21.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:21.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:21.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.715 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 63.627724409000002, 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:21.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:21.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:21.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:21.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:21.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:21.723 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:21.724 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:21.725 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:21.739+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:21.725 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:21.826 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:21.840+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:21.862 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:21.863 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:21.863 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:21.863 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:21.872 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:21.927 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:21.941+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:21.997 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:21.997 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:21.997 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:21.997 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:21.997 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:22.006 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:22.037 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:22.046 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:22.060+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:22.130 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:22.143+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:22.172 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:22.173 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.677122889000003, 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:22.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.677010682000002, 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.185 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:22.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:22.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:22.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:22.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:22.195 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676920444000004, 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:22.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:22.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:22.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:22.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:22.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676803801000005, 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.208 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:22.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:22.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:22.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:22.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:22.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:22.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676655432000004, 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:22.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:22.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:22.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:22.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:22.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:22.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:22.230 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676466063000007, 2020-11-16T22:24:22.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:22.232 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.232 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.232 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676252141000006, 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:22.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:22.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:22.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.676053240000002, 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.657 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:22.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:22.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:22.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:22.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.675872002999995, 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.667 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.668 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:22.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:22.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:22.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:22.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:22.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.675244688999996, 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:22.879 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:22.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:22.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:22.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:22.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.888 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:22.889 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:22.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:22.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:22.889 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:22.889 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:23.090 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.674947345999996, 2020-11-16T22:24:23.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:23.090 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.090 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:23.090 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:23.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:23.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:23.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:23.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.099 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:23.100 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.302 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:23.302 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:23.302 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.674540534000002, 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:23.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:23.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:23.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:23.310 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.674441315999999, 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:23.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:23.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:23.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:23.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:23.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:23.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.674246350000004, 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:23.523 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:23.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:23.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:23.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:23.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.533 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:23.734 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:23.734 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.673685277999994, 2020-11-16T22:24:23.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:23.734 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:23.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:23.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:23.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:23.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:23.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:23.944 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.673526796999994, 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:23.945 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:23.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:23.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:23.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:23.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:23.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:24.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:24.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:24.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:24.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:24.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:24.155 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.155 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:24.358 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.673393885999999, 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:24.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:24.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:24.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:24.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:24.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:24.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:24.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:24.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:24.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:24.569 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.673225277, 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:24.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:24.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:24.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.437521515999997, 2020-11-16T22:24:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:24.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:24.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:24.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:24.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:24.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:24.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:24.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:24.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:24.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:24.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:25.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:25.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:25.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:25.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:25.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:25.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:25.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:25.197 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:25.197 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:25.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:25.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.198 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:25.198 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.437394488999999, 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:25.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:25.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:25.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:25.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:25.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:25.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:25.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:25.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:25.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:25.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:25.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:25.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:25.615 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:25.615 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:25.615 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:25.615 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:25.615 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:25.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:24:25.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:24:25.819 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:24:25.819 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:24:25.819 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:24:25.820 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:24:25.820 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:24:25.822 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:25.822 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:25.823 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:25.823 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:25.824 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:25.833+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:25.824 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:25.923 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:25.937+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:25.964 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:25.965 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:24:25.965 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:25.965 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:25.966 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:25.976 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:26.024 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.038+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.099 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.099 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.108 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:26.152 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:26.166+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:26.225 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:26.238+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:26.233 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.234 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.243 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:26.325 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:26.340+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:26.379 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.380 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.389 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:26.432 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.446+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.515 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.515 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.524 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:26.565 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.579+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.627 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:26.641+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:26.628 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~8s 2020-11-16T22:24:26.689 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.689 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.698 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:26.727 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.741+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.825 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.825 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.828 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.842+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.834 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:26.928 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:26.942+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:26.957 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:26.957 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:26.966 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:27.028 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:27.042+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:27.074 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:24:27.092 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:27.092 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:27.101 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:27.142 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:27.156+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:27.227 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:27.227 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:27.229 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:27.243+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:27.236 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:27.330 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:27.343+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:27.400 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:27.401 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:27.603 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:27.603 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:27.617+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:27.703 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:27.718+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:27.767 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:27.767 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:27.776 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:27.819 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:27.833+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:27.824 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:27.904 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:27.904 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:27.906 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:27.920+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:27.912 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:27.976 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:27.997 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:27.997 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:27.997 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:27.997 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:27.998 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:27.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:28.000 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:28.007 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:28.021+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:28.077 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:28.078 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:28.086 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:28.107 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:28.122+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:28.209 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:28.223+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:28.251 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:28.252 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:28.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:28.252 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:28.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.252 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 78.175986527000006, 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:28.253 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:28.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:28.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:28.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:28.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:28.261 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 78.064556846000002, 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:28.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:28.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:28.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:28.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:28.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:28.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:28.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:28.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:28.473 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.756120295000002, 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:28.474 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:28.475 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:28.475 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:28.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:28.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:28.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:28.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:28.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:28.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:28.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:28.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:28.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:28.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:28.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.756004994999998, 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:28.891 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:29.094 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:29.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:29.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:29.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:29.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:29.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:29.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:29.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:29.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:29.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:29.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.307 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:29.307 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:29.510 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755896234999994, 2020-11-16T22:24:29.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:29.510 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:29.510 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:29.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:29.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:29.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:29.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:29.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:29.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:29.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:29.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:29.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:29.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:29.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:29.723 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:29.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:29.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.926 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:29.926 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755813885999999, 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:29.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:29.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:29.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:29.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:29.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:29.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:30.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:30.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:30.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:30.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:30.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:30.646 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755726077999995, 2020-11-16T22:24:30.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:30.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:30.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:30.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:30.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:30.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:31.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:31.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755631539000007, 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:31.064 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:31.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:31.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:31.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:31.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:31.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:31.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:31.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:31.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:31.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:31.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:31.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:31.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:31.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.480 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755545353000002, 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:31.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:31.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:31.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:31.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:31.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:31.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:31.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:31.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:31.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:31.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:31.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:31.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:31.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:31.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.896 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.755444968000006, 2020-11-16T22:24:31.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:31.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:31.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:31.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:31.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:31.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:31.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:32.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:32.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:32.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:32.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:32.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:32.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:32.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:32.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:32.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.750331646999996, 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:32.318 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:32.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:32.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:32.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:32.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:32.731 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.750225474000004, 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:32.732 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:32.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:32.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:32.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:32.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:32.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:32.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:32.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:32.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:32.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:32.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.750051463999995, 2020-11-16T22:24:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749955185000005, 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:33.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:33.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:33.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:33.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:33.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:33.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:33.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:33.568 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749767482999999, 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:33.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:33.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:33.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749575528999998, 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:33.982 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:33.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:33.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:33.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:33.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:33.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749412965000005, 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:33.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:33.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:33.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:33.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:34.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:34.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:34.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:34.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:34.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:34.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:34.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:34.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:34.403 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749237102999999, 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:34.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:34.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:34.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:34.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:34.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:34.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:34.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:34.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:34.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:34.614 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.749069058000003, 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:34.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:34.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:34.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:34.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:34.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:34.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:34.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:34.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.748827927999997, 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:34.825 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:34.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:34.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:34.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:35.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:35.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:35.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:35.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:35.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:35.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:35.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:35.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:35.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:35.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:35.238 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:35.239 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:35.239 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:35.239 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:35.239 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:35.239 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:35.240 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:24:35.241 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:35.241 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:24:35.241 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:24:35.242 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:24:35.242 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:24:35.242 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:24:35.242 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:24:35.244 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.255+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.244 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:35.342 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:35.355+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:35.382 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:35.382 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:35.382 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:35.383 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:35.392 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:35.400 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:35.419 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:35.419 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:24:35.419 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:35.421 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:35.421 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:35.421 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:35.421 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:35.422 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:35.424 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:35.442 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.456+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.518 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:35.518 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:35.527 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:35.544 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:35.557+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:35.644 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.658+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.693 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:35.693 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:35.702 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:35.745 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.759+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.828 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:35.829 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:35.838 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:35.845 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.860+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.855 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:24:35.946 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:35.960+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:35.962 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:35.962 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:35.971 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:36.047 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.061+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.049 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~18s 2020-11-16T22:24:36.096 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.096 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.105 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:36.149 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.163+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.232 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.232 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.241 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:36.282 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.296+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.350 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:36.363+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:36.405 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.405 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.414 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:36.450 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:36.464+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:36.538 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.539 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.547 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:36.552 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.565+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.653 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.667+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.675 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.675 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.684 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:36.753 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.766+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.808 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.808 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.817 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:36.854 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:36.868+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:36.941 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:36.941 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:36.949 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:36.993 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.007+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.055 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:37.068+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:37.076 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:37.076 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:37.085 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:37.156 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:37.169+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:37.249 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:37.249 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:37.257 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.270+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.258 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:37.358 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.371+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.424 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:37.425 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:37.425 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:37.435 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:37.476 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.490+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.560 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.573+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.562 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:37.562 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:37.571 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:37.577 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:37.596 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:37.598 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:37.599 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:37.660 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:37.673+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:37.695 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:37.696 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:37.704 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:37.761 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.775+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.861 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:37.876+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:37.866 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:24:37.870 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 87.794592996999995, 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 87.683163316000005, 2020-11-16T22:24:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374726765000005, 2020-11-16T22:24:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374611465000001, 2020-11-16T22:24:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374502704999998, 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374420356000002, 2020-11-16T22:24:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374332547999998, 2020-11-16T22:24:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374238008999995, 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374151823000005, 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.374051437999995, 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:37.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:37.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368938116999999, 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:37.967 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:37.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:37.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:37.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.974 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368831943999993, 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:37.975 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:37.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.982 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368657933999998, 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:37.983 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:37.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:37.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.990 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368561654999993, 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:37.991 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:37.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:37.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:37.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.998 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368373953000003, 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:37.999 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:38.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368181999000001, 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.006 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:38.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.368019434999994, 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.014 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:38.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:38.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:38.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.367843573000002, 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:38.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.367675528000007, 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.032 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:38.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.367434398, 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:38.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:38.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:38.048 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:38.049 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:38.063+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:38.049 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:38.150 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:38.163+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:38.177 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:38.178 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:38.187 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:38.251 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:38.265+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:38.312 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:38.312 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:38.313 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:38.313 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:38.313 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:38.321 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:38.362 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:38.376+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:38.453 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:38.466+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:38.489 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:38.489 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:38.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:38.489 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:38.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993688637999995, 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.490 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:38.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:38.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:38.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:38.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:38.500 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:38.501 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993576430999994, 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:38.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:38.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:38.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:38.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:38.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.512 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993486192999995, 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:38.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:38.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:38.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:38.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993369549999997, 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:38.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:38.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:38.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:38.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.535 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993221180999996, 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:38.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:38.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:38.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:38.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:38.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.993031811999998, 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.547 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:38.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:38.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:38.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:38.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:38.557 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.992817889999998, 2020-11-16T22:24:38.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:38.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:38.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:38.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:38.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:38.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.569 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.992618988999993, 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:38.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:38.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:38.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:38.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.992437752000001, 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:38.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:38.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:38.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:38.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.590 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.590 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:38.590 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.991810438000002, 2020-11-16T22:24:38.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:38.590 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:38.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:38.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:38.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:38.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.600 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.991513095000002, 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:38.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:38.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:38.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.991106282999993, 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.991007065000005, 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.619 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:38.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:38.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.990812098999996, 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.629 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:38.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:38.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:38.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.990251026999999, 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:38.639 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:38.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.645 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.990092546, 2020-11-16T22:24:38.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:38.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.653 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.989959635000005, 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:38.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:38.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.989791026000006, 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:38.661 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:38.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:38.667 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.754087265000003, 2020-11-16T22:24:38.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:38.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:38.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:38.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:38.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:38.679 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.753960237999998, 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:38.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:38.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:38.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:38.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:38.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:38.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:38.690 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:38.691 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:38.692 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:38.692 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:38.705+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:38.793 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:38.807+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:38.821 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:38.822 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:38.831 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:38.862 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:24:38.893 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:38.906+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:38.955 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:38.956 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:38.964 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:38.994 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:39.008+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:39.094 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:39.107+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:39.129 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:39.130 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:39.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:39.130 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:39.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.130 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 89.053999434999994, 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:39.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:39.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 88.942569754000004, 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.139 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:39.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:39.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:39.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.146 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.634133203000005, 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:39.147 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:39.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:39.156 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.634017903, 2020-11-16T22:24:39.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:39.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:39.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:39.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:39.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:39.167 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633909142999997, 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:39.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:39.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:39.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.177 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633826794000001, 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:39.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:39.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633738985999997, 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.188 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:39.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:39.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:39.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:39.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:39.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:39.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:39.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633644446999995, 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.202 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:39.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:39.211 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633558261000005, 2020-11-16T22:24:39.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:39.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:39.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:39.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:39.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:39.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:39.222 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.633457875999994, 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:39.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:39.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:39.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:39.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.628344554999998, 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.233 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:39.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:39.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:39.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.628238382000006, 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:39.241 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.248 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.628064371999997, 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:39.249 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.250 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.250 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:39.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:39.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:39.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.664 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627968093000007, 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:39.665 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:39.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:39.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:39.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:39.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:39.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:39.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:40.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.078 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627780391000002, 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:40.079 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:40.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:40.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:40.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:40.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:40.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:40.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:40.288 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627588437, 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:40.289 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:40.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:40.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:40.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:40.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:40.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:40.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:40.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627425873000007, 2020-11-16T22:24:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:40.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:40.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:40.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:40.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:40.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627250011000001, 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:40.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:40.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:40.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:40.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:40.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.627081966000006, 2020-11-16T22:24:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:41.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:41.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:41.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:41.539 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.626840836, 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:41.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:41.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:41.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:41.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:41.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:41.751 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:41.751 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:41.752 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:41.752 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:24:41.752 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:24:41.753 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:24:41.753 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2020-11-16T22:24:41.753 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:24:41.755 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:41.766+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:41.755 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:41.853 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:41.867+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:41.891 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:41.892 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:41.892 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:41.892 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:41.901 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:41.942 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:41.953 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:41.967+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:41.963 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:41.963 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:41.964 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:41.964 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:41.964 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:41.966 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:41.966 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:42.025 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.026 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.035 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:42.055 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.068+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.080 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:24:42.080 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:24:42.080 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 7.51665 -> 7.51665 (expected baseline 3.53553) 2020-11-16T22:24:42.080 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.0 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:24:42.080 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 18 -> 18 pgs (0.72 -> 0.72 * mean) 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0001 2020-11-16T22:24:42.081 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0001 2020-11-16T22:24:42.082 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:24:42.093 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:24:42.093 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2020-11-16T22:24:42.155 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:42.169+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:42.159 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.159 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.169 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:42.256 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.269+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.306 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.306 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.315 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:42.358 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.371+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.380 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:24:42.380 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 7.51665 -> 7.51665 (expected baseline 3.53553) 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.0 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 18 -> 18 pgs (0.72 -> 0.72 * mean) 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:24:42.381 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:24:42.382 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0001 2020-11-16T22:24:42.382 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0006 2020-11-16T22:24:42.382 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:24:42.393 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-11-16T22:24:42.393 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2020-11-16T22:24:42.459 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.472+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.480 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.480 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.489 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:42.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~24s 2020-11-16T22:24:42.572 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.585+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.654 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.654 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.660 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.673+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.663 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:42.681 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 7.51665 -> 7.51665 (expected baseline 3.53553) 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.0 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 18 -> 18 pgs (0.72 -> 0.72 * mean) 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:24:42.682 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:24:42.683 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:24:42.683 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:24:42.683 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0001 2020-11-16T22:24:42.683 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0001 2020-11-16T22:24:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:24:42.694 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:24:42.694 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2020-11-16T22:24:42.761 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.773+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.791 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.791 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.800 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:42.861 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:42.875+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:42.874 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:24:42.925 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:42.925 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:42.934 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:42.962 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:42.976+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:42.989 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:24:42.989 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:24:42.989 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 7.51665 -> 7.51665 (expected baseline 3.53553) 2020-11-16T22:24:42.989 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.0 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 18 -> 18 pgs (0.72 -> 0.72 * mean) 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0001 2020-11-16T22:24:42.990 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0001 2020-11-16T22:24:42.991 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:24:43.002 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:24:43.002 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2020-11-16T22:24:43.063 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.077+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:43.099 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.099 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.108 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:43.164 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:43.179+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:43.235 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.235 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.244 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:43.265 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:43.280+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 7.51665 -> 7.51665 (expected baseline 3.53553) 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.0 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 18 -> 18 pgs (0.72 -> 0.72 * mean) 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:24:43.289 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:24:43.290 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:24:43.290 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:24:43.290 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0001 2020-11-16T22:24:43.290 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0001 2020-11-16T22:24:43.291 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:24:43.366 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:43.381+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:43.368 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.369 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.378 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:43.467 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.480+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:43.502 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.503 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.512 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:43.568 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.581+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:43.638 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.638 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.647 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:43.669 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.682+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:43.769 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.783+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:43.771 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.771 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.781 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:43.870 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:43.884+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:43.907 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:43.907 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:43.916 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:43.968 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:43.970 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:43.983+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:44.043 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:44.043 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:44.052 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:44.071 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:44.086+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:44.114 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:44.133 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:44.133 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:44.133 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:44.134 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:44.134 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:44.135 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:44.136 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:44.172 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:44.185+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:44.217 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:44.218 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:44.218 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:44.218 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:44.218 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:44.227 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:44.273 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:44.286+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:44.374 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:44.388+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:44.394 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:44.394 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:44.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:44.394 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:44.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 94.318135319999996, 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:44.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:44.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:44.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.402 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 94.206705639000006, 2020-11-16T22:24:44.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:44.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:44.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:44.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:44.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:44.411 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.898269088000006, 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:44.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:44.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:44.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:44.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:44.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:44.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:44.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:44.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:44.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:44.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:44.623 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.898153788000002, 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:44.827 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:44.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:44.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:44.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:45.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:45.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:45.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:45.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:45.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:45.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:45.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:45.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:45.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:45.242 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:45.242 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.898045027999999, 2020-11-16T22:24:45.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:45.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:45.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:45.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:45.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:45.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:45.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:45.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:45.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:45.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:45.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:45.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:45.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:45.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:45.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:45.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:45.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.897962679000003, 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:45.660 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:45.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:45.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:45.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:45.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:45.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:45.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:45.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:45.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:45.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:45.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:45.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:45.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:46.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.076 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.897874870999999, 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:46.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:46.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:46.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:46.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:46.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:46.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:46.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:46.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:46.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:46.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:46.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:46.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:46.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:46.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:46.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.496 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.897780331999996, 2020-11-16T22:24:46.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:46.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:46.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:46.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:46.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:46.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:46.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.897694146000006, 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:46.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:46.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:46.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:46.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:46.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:46.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:46.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:46.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:46.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:47.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:47.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:47.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:47.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:47.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:47.126 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:47.127 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.897593760999996, 2020-11-16T22:24:47.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:47.127 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:47.127 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:47.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:47.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:47.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:47.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:47.543 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.543 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:47.543 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:47.746 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.89248044, 2020-11-16T22:24:47.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:47.746 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:47.746 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:47.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:47.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:47.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:47.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:47.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:47.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:47.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:47.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.892374266999994, 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:47.958 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:47.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:48.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:48.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:48.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:48.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:48.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:48.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:48.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:48.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.371 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.892200256999999, 2020-11-16T22:24:48.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:48.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:48.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:48.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:48.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:48.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:48.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.583 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.892103977999994, 2020-11-16T22:24:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:48.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:48.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:48.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:48.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.891916276000003, 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:48.998 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:48.999 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:48.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:48.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:48.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:49.410 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.891724322000002, 2020-11-16T22:24:49.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:49.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:49.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:49.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:49.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:49.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:49.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:49.621 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.891561757999995, 2020-11-16T22:24:49.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:49.623 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:49.623 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:49.623 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:49.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:49.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:49.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:49.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:49.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:49.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:49.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:49.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:49.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:49.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:49.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:50.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:50.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:50.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:50.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.036 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.891385896000003, 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:50.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:50.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:50.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:50.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:50.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:50.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.891217850999993, 2020-11-16T22:24:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.890976721000001, 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:50.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:50.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:50.469 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:50.469 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:50.469 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:24:50.470 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:50.471 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:24:50.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:24:50.471 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2020-11-16T22:24:50.471 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.2 doing powercycle of ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:24:50.472 INFO:teuthology.orchestra.console:Power on smithi122 2020-11-16T22:24:50.472 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:24:50.484 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:50.484+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:50.485 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:50.495 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:24:50.572 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:50.585+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:50.624 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:50.624 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:50.624 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:50.624 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:50.634 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:50.652 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:50.672 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:50.686+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:50.673 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:50.674 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:50.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:50.773 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:50.787+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:50.798 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:50.799 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:50.809 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:50.891 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:50.905+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:50.901 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:24:50.974 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:50.975 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:50.975 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:50.988+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:50.984 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:51.075 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.089+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.120 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.120 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.130 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:51.176 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:51.190+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:51.254 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.255 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.264 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:51.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2020-11-16T22:24:51.305 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:51.319+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:51.378 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.392+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.428 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.428 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.437 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:51.480 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.494+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.563 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.564 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.572 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:51.613 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.627+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.681 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.695+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.736 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.737 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.746 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:51.786 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.800+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.883 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:51.896+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:51.909 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:51.910 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:51.919 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:51.984 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:51.998+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.047 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.048 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.057 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:52.098 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:52.112+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.186 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:52.199+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:52.221 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.221 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.230 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:52.286 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:52.300+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.355 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.355 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.364 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:52.405 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:52.419+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.488 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:52.501+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:52.490 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.490 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.499 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:52.588 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:52.602+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.663 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.664 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.673 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:52.679 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:24:52.716 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:52.730+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:52.790 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:52.804+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:52.797 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:52.798 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:52.807 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:52.831 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:52.851 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:52.851 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:24:52.851 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:24:52.851 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:52.852 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:52.853 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:52.854 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:52.891 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:52.905+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:52.913 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:24:52.936 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:52.936 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:52.945 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:52.992 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:53.006+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:53.092 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:53.106+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:53.109 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:53.109 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:53.109 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:24:53.110 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:24:53.110 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:53.118 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:53.201 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:24:53.215+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:24:53.284 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 103.208608727, 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:24:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:24:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:24:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 103.09717904599999, 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:24:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:24:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:24:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788742494999994, 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:24:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:24:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:24:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:24:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788627195000004, 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:24:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:24:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:24:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:24:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:24:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788518435, 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:24:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:24:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:24:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:24:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788436086000004, 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:24:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:24:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:24:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:24:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788348278000001, 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:24:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:24:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:24:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788253738999998, 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:24:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:24:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:24:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788167552999994, 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:24:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:24:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:24:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.788067167999998, 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:24:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:24:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782953847000002, 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:24:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:24:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:24:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782847673999996, 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:24:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:24:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:24:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:24:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782673664000001, 2020-11-16T22:24:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:24:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:24:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:24:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782577384999996, 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:24:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:24:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782389683000005, 2020-11-16T22:24:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:24:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:24:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782197729000004, 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:24:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.782035164999996, 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:24:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:24:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:24:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.781859303000005, 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:24:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:24:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:24:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.781691257999995, 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:24:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:24:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:24:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:24:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 95.781450128000003, 2020-11-16T22:24:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:24:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:24:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:53.866 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:24:53.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:24:53.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:53.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:53.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:24:53.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:24:53.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:24:53.873 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:53.874 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:53.888+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:53.875 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:24:53.925 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:24:53.974 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:53.988+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:54.012 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:54.012 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:54.013 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:24:54.013 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:54.022 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:24:54.104 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:54.118+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:54.176 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:54.190+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:54.186 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:54.187 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:24:54.187 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:24:54.187 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:24:54.187 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:54.196 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:24:54.279 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:54.293+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:54.363 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:24:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:24:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867825529000001, 2020-11-16T22:24:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:24:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:24:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:24:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:24:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867713322, 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:24:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:24:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:24:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:24:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867623084000002, 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:24:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:24:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:24:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:24:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867506441000003, 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:24:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:24:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:24:54.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:24:54.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:54.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:24:54.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.409 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:54.410 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867358072000002, 2020-11-16T22:24:54.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:24:54.614 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.614 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.615 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:54.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:24:54.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:24:54.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:54.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:24:54.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:54.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:24:54.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.626 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.867168703000004, 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.627 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:24:54.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:24:54.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:54.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:24:54.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:24:54.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:54.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:24:54.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:24:54.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:24:54.841 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:54.842 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.866954781000004, 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:54.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:24:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:24:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:24:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:24:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:24:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:24:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:24:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:24:55.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:55.462 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:55.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.866755879999999, 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:24:55.463 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:55.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:24:55.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:24:55.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:55.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:24:55.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:24:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:55.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:24:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:24:55.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:55.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:24:55.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:24:55.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:55.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:24:55.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:24:55.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.866574643000007, 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:24:55.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:55.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:55.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:55.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:24:55.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:24:55.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:56.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:24:56.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:24:56.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:24:56.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:56.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:24:56.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:24:56.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:56.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:24:56.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:56.296 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.865947328999994, 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:56.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:24:56.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:56.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:24:56.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:24:56.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:24:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:24:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:24:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:24:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:56.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:24:56.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:24:56.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:56.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.712 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.865649985999994, 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:56.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:56.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:24:56.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:24:56.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:56.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:24:56.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:24:56.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:24:56.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:56.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:24:56.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:24:56.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:56.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:56.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:57.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:24:57.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:24:57.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:24:57.128 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:57.129 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.865243174, 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:57.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:57.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:57.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:57.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:24:57.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:24:57.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:57.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:24:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:57.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:24:57.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:24:57.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:57.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.865143955999997, 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:57.544 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:57.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:24:57.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:57.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:24:57.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:24:57.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:57.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:24:57.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:57.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:24:57.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:57.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:24:57.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:24:57.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:57.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.960 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.864948990000002, 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:24:57.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:24:58.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:58.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:24:58.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:24:58.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:24:58.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:58.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:24:58.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:58.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.864387918000006, 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:24:58.377 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:58.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:58.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:24:58.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:24:58.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:58.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:24:58.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:24:58.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:58.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:24:58.587 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:58.588 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:58.790 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:58.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.864229437000006, 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:58.791 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:58.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:24:58.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:58.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:24:58.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.864096525999997, 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:59.002 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:24:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:24:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:24:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:24:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:24:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.863927916999998, 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:24:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:24:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 77.628224156000002, 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:24:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:24:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:24:59.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:24:59.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:24:59.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:24:59.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:59.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:59.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.635 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 77.628097128999997, 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:24:59.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:24:59.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:24:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:24:59.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:24:59.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.647 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:24:59.647 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:24:59.647 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:24:59.647 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:24:59.656 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:24:59.657 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:24:59.659 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:24:59.660 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:59.662+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:59.749 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:59.763+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:59.798 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:24:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:24:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:24:59.799 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:24:59.808 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:24:59.818 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:24:59.838 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:24:59.838 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:24:59.839 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:24:59.839 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:24:59.839 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:24:59.842 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:24:59.842 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:24:59.849 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:59.863+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:59.859 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2020-11-16T22:24:59.933 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:24:59.933 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:24:59.942 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:24:59.950 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:24:59.964+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:24:59.960 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi122 2020-11-16T22:24:59.960 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi122 2020-11-16T22:25:00.019 DEBUG:teuthology.orchestra.console:expect: smithi122 login 2020-11-16T22:25:00.052 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:00.066+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:00.066 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:00.067 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:00.076 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:00.153 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:00.166+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:00.213 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:00.214 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:00.224 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:00.254 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:00.267+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:00.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:00.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:00.779 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:00.779 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:00.780 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:00.780 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:00.793+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:00.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~42s 2020-11-16T22:25:00.880 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:00.894+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:00.938 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:25:00.946 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:00.946 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:00.955 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:00.996 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.010+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.081 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:01.095+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:01.092 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.092 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.101 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:01.184 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:01.198+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:01.266 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.266 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.275 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:01.316 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:01.330+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:01.383 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.397+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.439 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.439 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.449 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:01.493 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.507+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.577 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.578 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.585 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.598+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.587 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:01.686 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.699+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.751 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.752 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.761 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:01.786 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:01.800+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:01.843 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:01.887 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:01.901+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:01.935 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:01.988 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:02.001+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:01.996 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:02.015 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:02.016 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:02.016 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:02.016 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:02.016 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:02.018 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:02.018 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:02.062 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:02.062 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:02.063 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:02.071 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:02.088 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:02.102+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:02.189 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:02.203+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:02.195 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:02.195 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:02.195 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:02.195 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:02.196 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:02.204 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:02.289 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:02.303+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:02.371 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:02.371 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:02.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:02.371 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:02.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 112.294905779, 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:02.372 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:02.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:02.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:02.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:02.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:02.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:02.580 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 112.183476098, 2020-11-16T22:25:02.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:02.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:02.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:02.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:02.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:02.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:02.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:02.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:02.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:02.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:02.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:02.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:02.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:02.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:02.999 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.875039547, 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:03.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:03.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:03.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:03.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:03.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:03.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:03.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:03.416 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.874924247, 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:03.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:03.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:03.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:03.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:03.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.87481548700001, 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:04.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:04.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:04.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:04.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:04.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:04.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:04.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:04.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:04.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:04.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:04.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:04.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:04.249 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.874733138, 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:04.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:04.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:04.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:04.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:04.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:04.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:04.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:04.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:04.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:04.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:04.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:04.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.87464533000001, 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:04.666 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:04.667 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:04.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:04.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:04.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:04.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:04.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:04.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:04.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:04.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:04.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:04.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:04.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:05.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:05.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:05.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:05.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:05.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:05.081 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:05.082 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:05.083 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.874550791, 2020-11-16T22:25:05.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:05.083 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:05.083 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:05.286 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:05.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:05.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:05.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:05.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:05.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:05.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:05.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:05.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:05.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:05.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:05.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.499 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.874464605, 2020-11-16T22:25:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:05.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:05.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:05.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:05.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:05.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:05.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:05.915 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:05.915 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:05.915 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:06.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.118 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:06.118 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.87436422, 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:06.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:06.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:06.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:06.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:06.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:06.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:06.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:06.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:06.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:06.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:06.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:06.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:06.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:06.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:06.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.86925089899999, 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:06.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:06.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:06.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:06.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:06.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:06.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:06.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:06.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:06.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.869144726, 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:06.747 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:06.950 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:06.950 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:06.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:06.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:06.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:06.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:06.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:06.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:06.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:06.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:06.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:07.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:07.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.160 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.86897071600001, 2020-11-16T22:25:07.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:07.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:07.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:07.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:07.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:07.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:07.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:07.575 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.868874437, 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:07.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:07.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:07.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:07.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:07.990 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:07.990 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.868686735, 2020-11-16T22:25:07.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:07.990 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:07.990 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:07.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:07.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:07.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:07.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:07.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:07.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:07.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:08.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:08.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.868494781, 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:08.200 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:08.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:08.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:08.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:08.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:08.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:08.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:08.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:08.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.412 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:08.412 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.868332217, 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:08.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:08.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:08.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:08.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:08.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:08.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.421 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.868156355, 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:08.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:08.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:08.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:08.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:08.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.86798831, 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:08.430 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:08.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:08.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:08.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.437 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.86774717999999, 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:08.438 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:08.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:08.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:08.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:08.446 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:08.446 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:08.447 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:25:08.447 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:08.449 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:08.462+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:08.450 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:08.548 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:08.562+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:08.587 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:08.587 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:08.587 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:08.597 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:08.608 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:08.629 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:08.630 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:08.630 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:08.630 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:08.630 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:08.632 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:08.633 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:08.649 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:08.663+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:08.721 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:08.721 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:08.730 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:08.774 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:08.788+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:08.851 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:08.865+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:08.855 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:08.855 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:08.864 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:08.952 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:08.966+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:08.961 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:25:09.002 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.002 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.011 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:09.055 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:09.069+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:09.136 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.137 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.145 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:09.153 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.166+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.253 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.267+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~51s 2020-11-16T22:25:09.310 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.310 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.319 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:09.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:09.369+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:09.446 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.447 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.455 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:09.469+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:09.456 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:09.555 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.569+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.581 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.581 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.590 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:09.673 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.687+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.756 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.756 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.758 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.772+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.765 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:09.858 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.872+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:09.891 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:09.892 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:09.901 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:09.959 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:09.972+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:10.025 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.025 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.034 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:10.077 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.091+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.158 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.159 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.161 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.175+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.168 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:10.262 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:10.275+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:10.295 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.296 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.304 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:10.362 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.377+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.463 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.478+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.469 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.469 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.478 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:10.564 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.578+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.603 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.603 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.612 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:10.634 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:10.664 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:10.678+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:10.740 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:10.740 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:10.749 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:10.785 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:10.793 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.807+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.805 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:10.805 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:10.805 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:10.805 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:10.806 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:10.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:10.808 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:10.866 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:10.880+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:10.874 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:10.874 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:10.874 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:10.874 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:10.874 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:10.883 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:10.967 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:10.981+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:10.974 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:25:11.049 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:11.050 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 120.974013145, 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:11.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:11.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:11.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:11.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:11.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:11.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:11.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.058 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 120.862583464, 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:11.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:11.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:11.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.554146913, 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:11.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:11.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.55403161300001, 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:11.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:11.902 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.902 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.553922853, 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:11.903 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:11.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:11.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:11.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:11.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:11.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:11.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:12.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:12.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.115 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.55384050399999, 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:12.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:12.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:12.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:12.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:12.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:12.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:12.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.328 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.553752696, 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:12.329 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:12.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:12.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:12.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:12.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:12.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:12.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:12.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:12.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:12.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:12.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:12.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:12.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.744 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.553658157, 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:12.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:12.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:12.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:12.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:12.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:12.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:12.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.956 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.553571971, 2020-11-16T22:25:12.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:12.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:12.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:13.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:13.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:13.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:13.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:13.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:13.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:13.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:13.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:13.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:13.170 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.553471586, 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:13.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:13.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:13.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:13.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:13.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:13.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:13.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:13.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:13.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:13.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:13.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:13.586 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.548358265, 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:13.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:13.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:13.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:13.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:13.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:13.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:13.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:13.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.548252092, 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:13.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:13.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:13.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:13.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.548078082, 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:14.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:14.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:14.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:14.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.219 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:14.219 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:14.219 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.547981803, 2020-11-16T22:25:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:14.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:14.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:14.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:14.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:14.633 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.54779410099999, 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:14.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:14.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:14.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:14.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:14.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:14.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:14.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:14.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:14.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:14.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.843 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:14.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:14.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.046 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:15.046 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:15.046 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.54760214700001, 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:15.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:15.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:15.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:15.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:15.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:15.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:15.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:15.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.547439583, 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:15.258 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:15.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:15.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:15.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:15.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:15.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:15.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:15.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:15.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:15.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:15.672 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.54726372099999, 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:15.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:15.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:15.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:15.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:15.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:15.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:15.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:15.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:15.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:15.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:15.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:15.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:16.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.086 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:16.086 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.547095676, 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:16.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:16.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:16.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:16.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:16.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:16.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:16.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:16.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:16.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:16.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:16.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:16.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.298 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:16.299 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.299 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:16.299 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:16.300 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 113.54685454600001, 2020-11-16T22:25:16.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:16.300 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:16.300 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:16.300 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:16.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:16.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:16.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:16.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:16.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:16.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:16.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:16.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:16.712 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:16.713 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:16.715 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:25:16.716 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:16.728+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:16.716 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:16.814 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:16.828+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:16.853 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:16.853 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:16.853 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:16.854 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:16.863 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:16.906 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:16.927 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:16.930 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:17.134 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:17.135 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:16.959+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:17.135 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:17.149+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:17.136 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.136 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.136 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:17.236 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:17.250+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:17.301 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.301 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.310 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:17.337 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:17.352+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:17.438 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:17.452+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:17.447 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.447 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.457 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:17.539 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:17.553+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:17.584 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.584 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.593 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:17.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~59s 2020-11-16T22:25:17.676 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:17.690+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:17.741 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:17.755+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:17.758 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.758 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.767 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:17.841 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:17.856+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:17.893 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:17.893 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:17.902 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:17.942 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:17.957+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:18.027 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.027 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.036 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:18.079 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.093+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.143 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.157+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.161 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.161 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.170 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:18.244 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:18.258+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:18.299 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.299 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.308 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:18.345 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.359+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.446 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.461+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.471 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.472 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.481 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:18.547 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:18.561+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:18.606 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.606 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.615 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:18.658 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.672+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.716 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:25:18.741 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.741 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.748 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:18.763+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:18.750 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:18.849 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.862+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:18.915 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:18.916 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:18.924 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:18.965 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:18.979+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:19.050 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:19.065+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:19.089 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:19.089 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:19.098 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:19.135 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:19.151 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:19.165+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:19.224 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:19.224 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:19.233 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:19.274 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:19.288+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:19.297 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:19.297 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:19.352 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:19.367+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:19.398 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:19.398 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:19.398 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:19.398 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:19.398 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:19.407 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:19.454 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:19.468+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:19.534 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.45797462199999, 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:19.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:19.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:19.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:19.950 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.34654494099999, 2020-11-16T22:25:19.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:19.950 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:19.950 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:19.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:19.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:20.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:20.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:20.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:20.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:20.160 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:20.161 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03810839, 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:20.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:20.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:20.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:20.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:20.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:20.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:20.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:20.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:20.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:20.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:20.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:20.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.581 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03799309, 2020-11-16T22:25:20.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:20.583 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:20.583 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:20.583 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:20.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:20.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:20.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:20.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:20.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:20.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:20.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:20.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:20.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:20.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:20.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:20.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:20.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03788433, 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:20.999 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:21.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:21.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:21.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:21.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:21.618 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:21.618 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.037801981, 2020-11-16T22:25:21.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:21.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:21.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:21.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:21.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:21.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:21.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.034 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:22.034 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.034 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.037714173, 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:22.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:22.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:22.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:22.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:22.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:22.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:22.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:22.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:22.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:22.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:22.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:22.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:22.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:22.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:22.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:22.451 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.037619634, 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:22.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:22.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:22.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:22.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:22.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:22.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:22.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:22.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:22.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:22.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:22.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:22.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:22.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:22.867 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.037533448, 2020-11-16T22:25:22.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:22.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:22.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:22.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:22.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:22.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:22.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:23.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:23.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:23.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:23.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:23.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:23.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:23.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:23.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:23.283 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:23.284 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03743306299999, 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:23.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:23.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:23.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:23.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:23.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:23.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:23.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:23.497 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.032319742, 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:23.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:23.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:23.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:23.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:23.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.505 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03221356900001, 2020-11-16T22:25:23.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:23.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:23.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.513 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.032039559, 2020-11-16T22:25:23.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:23.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:23.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.521 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03194327999999, 2020-11-16T22:25:23.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:23.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:23.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:23.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:23.530 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.031755578, 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:23.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:23.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:23.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.031563624, 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.537 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:23.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:23.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:23.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.544 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03140105999999, 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.545 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.031225198, 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.03105715300001, 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.030816023, 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:23.577 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:23.577 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:23.579 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:25:23.580 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:23.592+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:23.580 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:23.679 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:23.692+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:23.718 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:23.718 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:23.727 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:23.770 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:23.780 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:23.794+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:23.789 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:23.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:23.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:23.790 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:23.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:23.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:23.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:23.852 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:23.852 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:23.861 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:23.905 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:23.919+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:23.982 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:23.996+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:23.987 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:23.987 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:23.996 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:24.083 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:24.096+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:24.132 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.133 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.142 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:24.184 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:24.198+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:24.285 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:24.298+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:24.306 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.307 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.316 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:24.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~66s 2020-11-16T22:25:24.388 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:24.402+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:24.480 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.480 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.489 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:24.502+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:24.489 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:24.590 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:24.604+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:24.617 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.617 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.626 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:24.709 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:24.723+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:24.791 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.791 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.793 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:24.807+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:24.800 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:24.894 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:24.908+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:24.965 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:24.965 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:24.974 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:24.995 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:25.008+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:25.095 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:25.108+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:25.099 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.099 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.108 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:25.196 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:25.209+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:25.272 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.272 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.281 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:25.297 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:25.311+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:25.397 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:25.411+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:25.446 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.446 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.455 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:25.498 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:25.512+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:25.580 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.580 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.582 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:25:25.589 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:25.630 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:25.644+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:25.700 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:25.714+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:25.754 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.754 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.763 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:25.793 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:25.800 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:25.814+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:25.888 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:25.888 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:25.897 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:25.934 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:25.941 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:25.954+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:25.953 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:25.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:25.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:25.954 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:25.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:25.956 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:25.957 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:26.002 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:26.016+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:26.005 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:25:26.025 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:26.025 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:26.025 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:26.025 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:26.034 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:26.117 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:26.130+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:26.198 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:26.199 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:26.204 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:26.218+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:26.208 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:26.304 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:26.317+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:26.374 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:26.374 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:26.375 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 136.29868766600001, 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:26.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:26.383 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 136.187257985, 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:26.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:26.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:26.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.391 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.878821434, 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:26.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:26.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:26.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:26.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.404 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.405 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.405 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.878706134, 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:26.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87859737400001, 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.416 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:26.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:26.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:26.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:26.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87851502500001, 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:26.426 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:26.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:26.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:26.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:26.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.878427217, 2020-11-16T22:25:26.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:26.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:26.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:26.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:26.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:26.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:26.446 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87833267799999, 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:26.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:26.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.456 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87824649199999, 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:26.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:26.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87814610699999, 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87303278600001, 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.872926613, 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87275260300001, 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87265632399999, 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87246862200001, 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87227666800001, 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87211410399999, 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.871938242, 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87177019699999, 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.87152906700001, 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:26.560 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:26.561 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:26.562 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:26.574+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:26.661 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:26.675+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:26.691 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:26.691 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:26.692 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:26.692 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:26.701 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:26.762 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:26.776+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:26.824 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:26.825 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:26.825 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:26.825 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:26.825 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:26.835 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:26.877 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:26.891+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:26.961 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.962 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46633406000001, 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:26.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:26.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:26.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46622185300001, 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:26.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:26.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:26.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.984 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46613161499999, 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:26.985 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:26.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:26.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:26.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:26.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:26.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:26.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:26.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:26.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:26.996 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46601497200001, 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:26.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:26.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:27.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:27.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:27.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.007 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46586660299999, 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:27.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:27.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:27.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:27.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:27.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.465677234, 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.019 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:27.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:27.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:27.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.029 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.465463312, 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:27.030 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:27.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:27.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:27.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:27.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:27.041 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46526441099999, 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:27.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:27.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:27.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:27.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.050 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.465083174, 2020-11-16T22:25:27.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:27.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:27.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:27.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.060 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46445585999999, 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:27.061 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:27.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:27.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:27.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46415851699999, 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.071 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:27.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:27.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:27.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:27.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.081 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46375170499999, 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:27.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:27.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:27.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.089 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46365248699999, 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:27.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.099 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46345752100001, 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:27.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.462896449, 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:27.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:27.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.462737968, 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:27.117 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:27.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:27.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:27.124 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46260505699999, 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:27.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:27.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:27.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.131 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 129.46243644800001, 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:27.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:27.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:27.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 110.22673268699999, 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.139 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:27.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:27.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:27.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:27.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.149 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 110.22660566, 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:27.150 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:27.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:27.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:27.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:27.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.161 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:27.161 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:27.161 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:27.162 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:27.176+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:27.162 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:27.262 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:27.275+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:27.295 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:27.295 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:27.295 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:27.295 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:27.304 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:27.387 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:27.400+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:27.464 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:27.478+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:27.468 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:27.469 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:27.469 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:27.469 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:27.469 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:27.477 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:27.565 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:27.578+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:27.645 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:27.645 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:27.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:27.645 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 137.56949209199999, 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:27.646 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:27.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.653 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 137.45806241099999, 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.661 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14962585999999, 2020-11-16T22:25:27.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:27.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.671 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14951056000001, 2020-11-16T22:25:27.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:27.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.681 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14940179999999, 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:27.682 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:27.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:27.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.691 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.149319451, 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.692 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:27.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:27.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:27.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:27.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14923164300001, 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.702 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:27.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:27.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:27.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:27.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:27.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.712 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.149137104, 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:27.713 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:27.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:27.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.722 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.149050918, 2020-11-16T22:25:27.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:27.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:27.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.732 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.148950533, 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:27.733 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:27.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:27.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:27.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.742 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14383721199999, 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.743 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:27.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:27.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:27.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14373103899999, 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:27.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14355702899999, 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:27.759 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:27.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:27.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:27.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14346075, 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:27.767 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:27.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:27.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:27.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.774 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.143273048, 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:27.775 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:27.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:27.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.982 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.982 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.143081094, 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.983 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:27.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:27.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:27.990 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:27.991 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14291853, 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:27.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:27.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:27.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:27.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:27.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.14274266800001, 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:28.000 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:28.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:28.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:28.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:28.007 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:28.008 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.142574623, 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:28.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:28.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:28.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:28.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.016 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.142333493, 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:28.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:28.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:28.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:28.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:28.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:28.024 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.025 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:28.025 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:28.025 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:28.025 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:28.026 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:25:28.027 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:28.040+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:28.028 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:28.127 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:28.141+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:28.164 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:28.165 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:28.165 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:28.165 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:28.174 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:28.199 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:28.222 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:28.223 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:28.223 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:28.223 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:28.223 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:28.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:28.225 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:28.227 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:28.241+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:28.298 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.299 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.307 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:28.351 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:28.365+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:28.430 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:28.444+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:28.433 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.433 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.442 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:28.531 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:28.546+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:28.572 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.573 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.581 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:28.632 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:28.646+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:28.705 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.706 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.714 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:28.732 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:28.747+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:28.833 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:28.846+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:28.839 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.839 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.848 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:28.934 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:28.948+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:28.976 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:28.976 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:28.986 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:29.009 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:25:29.069 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:29.083+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:29.136 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.149+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.151 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.151 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.161 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:29.237 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.250+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.325 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.325 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.334 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:29.338 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:29.352+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:29.439 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.452+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.461 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.461 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.470 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:29.554 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.568+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.634 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.635 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.640 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:29.654+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:29.644 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:29.740 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.753+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.808 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.808 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.817 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:29.858 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.872+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.942 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:29.956+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:29.944 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:29.944 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:29.953 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:30.043 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:30.057+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:30.077 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:30.078 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:30.086 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:30.169 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:30.183+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:30.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~72s 2020-11-16T22:25:30.226 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:30.244 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:30.258+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:30.251 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:30.251 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:30.260 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:30.345 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:30.359+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:30.378 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:30.387 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:30.387 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:30.396 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:30.397 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:30.400 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:30.400 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:30.445 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:30.460+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:30.546 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:30.561+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:30.560 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:30.560 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:30.560 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:30.560 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:30.560 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:30.569 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:30.647 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:30.661+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.696 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 140.620200098, 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:30.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:30.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:30.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.704 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 140.50877041699999, 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:30.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:30.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.712 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.20033386599999, 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:30.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:30.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:30.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.722 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.20021856599999, 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:30.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:30.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:30.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.732 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.200109806, 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:30.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:30.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:30.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:30.741 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.200027457, 2020-11-16T22:25:30.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:30.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:30.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:30.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:30.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19993964899999, 2020-11-16T22:25:30.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:30.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:30.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:30.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.761 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19984511000001, 2020-11-16T22:25:30.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:30.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.771 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19975892400001, 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:30.772 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:30.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:30.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.781 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19965853900001, 2020-11-16T22:25:30.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:30.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:30.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:30.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.194545218, 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.792 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:30.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:30.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:30.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.194439045, 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.800 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:30.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:30.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:30.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.194265035, 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.808 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:30.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:30.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19416875600001, 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.816 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:30.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:30.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19398105400001, 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:30.824 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:30.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:30.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:30.830 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.1937891, 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:30.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:30.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:30.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19362653600001, 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:30.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:30.846 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19345067399999, 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:30.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:30.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:30.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:30.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:30.854 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.19328262900001, 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:30.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:30.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:30.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:30.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.193041499, 2020-11-16T22:25:30.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:30.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:30.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:30.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:30.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.869 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.870 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.870 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:30.870 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:30.870 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:30.871 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:30.885+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:30.872 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:30.972 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:30.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:31.001 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:31.001 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:31.010 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:31.022 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:25:31.073 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:31.087+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:31.135 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:31.135 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:31.135 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:31.136 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:31.136 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:31.144 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:31.174 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:31.188+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:31.275 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:31.289+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:31.311 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:31.312 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81622153699999, 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:31.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:31.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:31.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:31.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:31.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:31.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:31.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81610932999999, 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:31.324 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:31.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:31.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:31.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:31.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:31.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:31.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.816019092, 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:31.335 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:31.336 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:31.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:31.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:31.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:31.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:31.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:31.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:31.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:31.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:31.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:31.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:31.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:31.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:31.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81590244899999, 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:32.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:32.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:32.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:32.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:32.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:32.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:32.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:32.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:32.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:32.372 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81575408, 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:32.373 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:32.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:32.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:32.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:32.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:32.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81556471100001, 2020-11-16T22:25:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:32.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:32.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:32.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:33.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:33.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:33.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81535078900001, 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:33.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:33.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:33.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.421 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.815151888, 2020-11-16T22:25:33.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:33.423 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:33.423 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:33.626 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:33.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:33.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:33.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:33.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:33.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:33.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81497065100001, 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.254 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.814343337, 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:34.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:34.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:34.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:34.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:34.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:34.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:34.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:34.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:34.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:34.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.814045994, 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:34.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:34.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:34.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.813639182, 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:35.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:35.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:35.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:35.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.099 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:35.302 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:35.302 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.813539964, 2020-11-16T22:25:35.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:35.302 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:35.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:35.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:35.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:35.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:35.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:35.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:35.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:35.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:35.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:35.515 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:35.515 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:35.718 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:35.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.718 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81334499799999, 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:35.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:35.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:35.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:35.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:35.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:35.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:35.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:35.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:35.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:35.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:35.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81278392600001, 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:36.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:36.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81262544500001, 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:36.143 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:36.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:36.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:36.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:36.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:36.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:36.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:36.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:36.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:36.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.555 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.812492534, 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:36.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:36.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:36.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:36.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:36.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:36.766 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:36.767 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.81232392499999, 2020-11-16T22:25:36.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:36.767 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 114.576620164, 2020-11-16T22:25:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 114.576493137, 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:37.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:37.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:37.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:37.814 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:37.815 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:37.815 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:37.815 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:37.815 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:37.815 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:37.816 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:37.818 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:25:37.818 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:37.830+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:37.819 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:37.917 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:37.931+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:37.958 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:37.958 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:37.967 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:38.007 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:38.028 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:38.032 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:38.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:38.050 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.064+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.119 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.133+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.132 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.133 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.142 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:38.220 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:38.234+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:38.307 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.307 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.316 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:38.320 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.334+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.421 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:38.435+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:38.453 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.453 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.462 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:38.546 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:38.560+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:38.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2020-11-16T22:25:38.622 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.636+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.627 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.627 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.636 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:38.723 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.737+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.801 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.802 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.811 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:38.855 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.869+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.926 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:38.940+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:38.941 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:38.941 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:38.950 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:39.033 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.047+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.114 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.115 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.123 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:39.127 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:39.141+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:39.228 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.242+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.288 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.289 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.297 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:39.341 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:39.355+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.429 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.443+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.434 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:39.530 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.544+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.598 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.598 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.607 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:39.631 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.645+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.732 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.732 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.732 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.746+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:39.741 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:39.819 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:25:39.833 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:39.847+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:39.868 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:39.868 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:39.877 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:39.934 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:39.948+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:40.001 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:40.001 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:40.010 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:40.034 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:40.035 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:40.049+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:40.136 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:40.150+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:40.174 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:40.175 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:40.176 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:40.184 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:40.196 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:40.196 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:40.196 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:40.196 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:40.197 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:40.198 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:40.199 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:40.237 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:40.251+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:40.311 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:40.312 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:40.312 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:40.312 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:40.320 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:40.365 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:40.378+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:40.439 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:40.453+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:40.445 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:40.445 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:40.445 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:40.445 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:40.445 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:40.454 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:40.540 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:40.554+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:40.621 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:40.622 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:40.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:40.622 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:40.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.622 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12617856899999, 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:40.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:40.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:40.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:40.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:40.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:40.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:40.633 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:40.634 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12606636199999, 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:40.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:40.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:40.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:40.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:40.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:40.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:40.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:40.849 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:40.850 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.125976124, 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:40.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:40.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:40.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:40.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:41.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:41.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:41.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:41.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:41.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:41.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12585948099999, 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.125711112, 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:41.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:41.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:41.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.102 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.102 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.102 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12552174300001, 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:42.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12530782100001, 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:42.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:42.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:42.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12510892, 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:42.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.541 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.742 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:42.742 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12492768300001, 2020-11-16T22:25:42.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:42.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:42.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:42.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:42.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:42.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:42.752 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.124300369, 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:42.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:42.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:42.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:42.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:42.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:42.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:42.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:42.963 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:42.964 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.124003026, 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:42.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:42.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:42.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:42.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:42.974 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.123596214, 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:42.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:42.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:42.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:43.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:43.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:43.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:43.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.123496996, 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:43.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:43.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:43.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:43.197 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12330202999999, 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:43.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:43.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:43.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12274095800001, 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:43.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:43.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:43.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.617 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12258247700001, 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:43.618 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:43.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.122449566, 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:43.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:43.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.633 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:43.633 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.12228095699999, 2020-11-16T22:25:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.840 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 123.886577196, 2020-11-16T22:25:43.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:43.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:43.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:43.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:43.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:43.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:43.852 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 123.886450169, 2020-11-16T22:25:44.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:44.054 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:44.054 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:44.054 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:44.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:44.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:44.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:44.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:44.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:44.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:44.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:44.267 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:44.268 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:44.268 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:44.270 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:25:44.270 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:44.283+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:44.271 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:44.370 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:44.384+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:44.408 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:44.409 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:44.409 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:44.418 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:44.461 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:44.480 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:44.481 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:44.481 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:44.481 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:44.481 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:44.483 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:44.483 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:44.501 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:44.515+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:44.572 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:44.586+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:44.583 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:44.583 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:44.593 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:44.674 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:44.687+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:44.758 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:44.758 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:44.767 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:44.811 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:44.825+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:44.876 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:44.889+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:44.904 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:44.904 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:44.914 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:44.997 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.011+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~87s 2020-11-16T22:25:45.078 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.091+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.079 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.079 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.089 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:45.179 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.193+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.253 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.254 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.263 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:45.304 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.318+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.381 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:45.395+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:45.390 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.390 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.399 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:45.482 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.496+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.563 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.564 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.572 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:45.615 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:45.629+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:45.684 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:45.698+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:45.697 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.697 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.706 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:45.784 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:45.799+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:45.833 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.833 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.842 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:45.885 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:45.899+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:45.966 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:45.966 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:45.975 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:46.018 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.032+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.086 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.100+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.100 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:46.100 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:46.109 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:46.187 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:46.201+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:46.235 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:46.236 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:46.244 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:46.272 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:25:46.288 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.301+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.389 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.402+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.408 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:46.409 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:46.417 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:46.484 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:46.490 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:46.504+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:46.542 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:46.542 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:46.551 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:46.592 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:46.606+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:46.625 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:46.645 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:46.648 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:46.648 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:46.679 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:46.679 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:46.688 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:46.691 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.705+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.792 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.806+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.812 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:46.813 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:46.813 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:46.813 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:46.813 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:46.822 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:46.904 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:46.918+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:46.989 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:46.989 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:46.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:46.989 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:46.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.493557052, 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:46.990 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:46.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:46.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:47.000 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.493444845, 2020-11-16T22:25:47.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:47.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:47.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:47.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:47.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:47.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.012 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49335460699999, 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:47.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:47.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:47.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:47.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:47.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.023 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.493237964, 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.024 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:47.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:47.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:47.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:47.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:47.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:47.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:47.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:47.237 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49308959499999, 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:47.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:47.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:47.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:47.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:47.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.451 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49290022599999, 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:47.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:47.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:47.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.665 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49268630399999, 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:47.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:47.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:47.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:47.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:47.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.676 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49248740300001, 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.677 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:47.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:47.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:47.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:47.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49230616599999, 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49167885200001, 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:48.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49138150900001, 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:48.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49097469700001, 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.122 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:48.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:48.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:48.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49087547900001, 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.490680513, 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.49011944099999, 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.48996095999999, 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.48982804900001, 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:48.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:48.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:48.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:48.574 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.578 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.48965944, 2020-11-16T22:25:48.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:48.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:48.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.586 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 130.253955679, 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:48.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:48.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:48.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:48.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:48.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:48.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:48.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:48.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.802 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 130.25382865200001, 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:48.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:48.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:48.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:48.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:48.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:48.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.813 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:48.814 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:48.814 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:48.815 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:48.816 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:25:48.817 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:48.829+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:48.817 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:48.916 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:48.930+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:48.953 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:48.953 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:48.954 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:48.954 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:48.962 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:49.003 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:49.017 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.030+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.023 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:49.023 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:49.023 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:49.024 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:49.024 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:49.026 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:49.026 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:49.087 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.087 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.096 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:49.118 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.132+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.219 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.233+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.221 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.221 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.230 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:49.320 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:49.334+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:49.367 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.367 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.376 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:49.421 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.434+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.501 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.501 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.510 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:49.521 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:49.535+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:49.622 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:49.636+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:49.675 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.676 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.684 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:49.723 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.737+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.811 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.812 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.820 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:49.823 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:49.837+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:49.923 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:49.937+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:49.945 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:49.945 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:49.954 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:50.023 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:50.037+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:50.078 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.078 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.087 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:50.125 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.139+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.222 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:50.226 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.240+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.327 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:50.340+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:50.347 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.348 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.357 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:50.428 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.442+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.522 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.522 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.528 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.542+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.531 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:50.629 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.643+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.658 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.659 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.668 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:50.751 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:50.765+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:50.818 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:25:50.831 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:50.845+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:50.832 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:50.833 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:50.841 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:50.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~92s 2020-11-16T22:25:50.932 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:50.945+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.006 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:51.006 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:51.015 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:51.027 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:51.059 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.073+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.133 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.147+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.146 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:51.147 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:51.156 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:51.176 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:51.195 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:51.196 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:51.196 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:51.196 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:51.196 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:51.198 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:51.199 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:51.235 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:51.249+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:51.321 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:51.321 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:51.330 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:51.373 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.387+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.437 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.451+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.456 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:51.456 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:51.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:51.456 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:51.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.96070206100001, 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.457 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:51.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:51.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:51.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:51.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.96058985400001, 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.469 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:51.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:51.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:51.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:51.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.96049961599999, 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.481 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:51.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:51.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:51.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:51.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.492 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.96038297300001, 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:51.493 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:51.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:51.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:51.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:51.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.96023460399999, 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.505 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:51.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:51.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:51.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:51.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.516 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.960045235, 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:51.517 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:51.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:51.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:51.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:51.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.528 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.959831313, 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.529 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:51.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:51.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:51.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:51.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.540 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95963241199999, 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:51.541 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:51.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:51.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:51.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.551 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.959451175, 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:51.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:51.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:51.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:51.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.561 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95882386100001, 2020-11-16T22:25:51.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:51.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:51.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:51.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.572 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95852651800001, 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:51.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:51.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:51.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:51.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.582 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95811970599999, 2020-11-16T22:25:51.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:51.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95802048799999, 2020-11-16T22:25:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:51.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:51.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:51.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:51.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:51.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:51.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:51.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:51.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:51.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:51.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:51.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:51.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:51.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:51.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:51.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:51.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:51.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:51.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:51.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:51.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:51.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:51.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:51.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:51.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:51.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.614 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.614 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.615 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.615 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:51.615 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95782552200001, 2020-11-16T22:25:51.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:51.616 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.616 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.616 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:51.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:51.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:51.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:51.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:51.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:51.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:51.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:51.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:51.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:51.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:51.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:51.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:51.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:51.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:51.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:51.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:51.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:51.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:51.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:51.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:51.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:51.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:51.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:51.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:51.633 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.634 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95726445, 2020-11-16T22:25:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.957105969, 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.95697305799999, 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.956804449, 2020-11-16T22:25:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:51.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:51.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 134.72110068800001, 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.669 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:51.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:51.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:51.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:51.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.680 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 134.72097366099999, 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:51.681 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:51.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:51.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:51.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:51.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:51.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.692 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.693 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:51.693 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:51.693 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:51.694 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:51.695 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:51.708+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:51.795 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.809+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.825 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:51.825 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:51.826 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:51.826 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:51.835 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:51.896 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:51.910+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:51.959 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:51.959 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:51.959 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:51.959 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:51.960 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:51.968 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:52.011 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:52.025+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:52.074 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:25:52.095 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:52.095 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 162.019628463, 2020-11-16T22:25:52.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:52.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:52.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:52.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:52.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.908198782, 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:52.105 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:52.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:52.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.113 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.599762231, 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:52.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:52.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:52.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:52.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.124 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.599646931, 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:52.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:52.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:52.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:52.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.134 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59953817100001, 2020-11-16T22:25:52.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:52.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:52.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:52.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:52.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.145 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59945582200001, 2020-11-16T22:25:52.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:52.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:52.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:52.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59936801399999, 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:52.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59927347499999, 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:52.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:52.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:52.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.178 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59918728900001, 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:52.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:52.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:52.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.188 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59908690399999, 2020-11-16T22:25:52.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:52.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:52.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:52.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:52.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:52.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:52.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:52.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:52.201 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59397358300001, 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:52.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:52.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:52.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:52.210 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59386741, 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:52.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:52.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:52.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59369340000001, 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.219 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:52.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:52.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:52.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59359712099999, 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.228 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:52.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:52.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:52.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:52.236 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59340941900001, 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:52.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:52.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.243 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59321746500001, 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:52.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:52.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:52.251 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:52.252 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59305490099999, 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:52.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:52.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:52.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.260 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.592879039, 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:52.261 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:52.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:52.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59271099399999, 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.270 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:52.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:52.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:52.277 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:52.278 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.59246986400001, 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:52.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:52.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:52.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:52.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:52.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:52.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:52.287 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:52.287 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:52.288 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:52.289 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:52.302+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:52.389 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:52.403+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:52.425 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:52.425 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:52.426 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:25:52.426 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:52.435 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:52.489 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:52.503+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:52.559 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:52.560 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:25:52.560 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:25:52.560 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:25:52.560 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:52.568 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:52.611 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:52.623+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:52.691 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:52.705+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:52.737 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.241946856, 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.241834649, 2020-11-16T22:25:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24174441100001, 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.241627768, 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:52.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:52.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.784 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24147939900001, 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:52.785 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:52.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.796 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24129002999999, 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:52.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:52.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:52.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:52.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:52.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.807 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24107610799999, 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:52.808 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:52.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:52.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:52.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:52.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.819 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24087720700001, 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:52.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:52.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:52.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:52.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.829 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24069596999999, 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:52.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:52.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:52.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:52.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:52.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.839 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.24006865600001, 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.840 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:52.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:52.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:52.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.849 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23977131300001, 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:52.850 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:52.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:52.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:52.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.859 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23936450100001, 2020-11-16T22:25:52.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.868 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23926528300001, 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:52.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:52.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:52.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:52.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.239070317, 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.879 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:52.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:52.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23850924499999, 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:52.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:52.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:52.895 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23835076399999, 2020-11-16T22:25:52.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:52.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:52.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.23821785300001, 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:52.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:52.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:52.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:52.911 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.238049244, 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:52.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:52.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 136.002345483, 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.919 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:52.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:52.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:52.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.929 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 136.00221845600001, 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:52.930 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:52.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:52.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:52.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:52.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:52.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:52.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:52.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:52.941 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:52.943 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:52.943 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:52.957+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:53.044 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:53.057+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:53.074 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:53.075 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:53.075 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:53.075 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:53.084 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:53.145 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:53.159+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:53.200 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:53.208 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:53.208 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:53.208 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:25:53.208 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:25:53.208 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:53.217 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:53.260 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:53.274+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:53.343 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:25:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 163.268088272, 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:25:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:25:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:25:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 163.156658591, 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:25:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:25:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84822204, 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:25:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:25:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:25:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84810673999999, 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:25:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84799798, 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:25:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:25:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:25:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84791563100001, 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:25:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:25:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:25:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:25:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84782782299999, 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:25:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:25:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:25:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:25:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84773328399999, 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:25:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:25:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:25:53.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:25:53.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.420 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84764709800001, 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:25:53.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:25:53.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:25:53.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:25:53.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.430 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84754671300001, 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:25:53.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:25:53.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:25:53.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:25:53.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:25:53.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.440 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.842433392, 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:25:53.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:25:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:25:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.842327219, 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:25:53.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:25:53.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:25:53.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.456 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.842153209, 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:25:53.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:25:53.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:25:53.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.464 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84205693000001, 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:25:53.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:25:53.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:25:53.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:25:53.472 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84186922800001, 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:25:53.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:25:53.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:25:53.478 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84167727400001, 2020-11-16T22:25:53.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:53.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:25:53.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:25:53.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:25:53.486 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84151471000001, 2020-11-16T22:25:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:53.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:25:53.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:25:53.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:25:53.494 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:53.495 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84133884799999, 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:53.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:25:53.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:25:53.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:25:53.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:53.503 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84117080300001, 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:53.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:25:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:25:53.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:25:53.510 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 155.84092967300001, 2020-11-16T22:25:53.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:53.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:53.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:25:53.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:25:53.518 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.519 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.519 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.519 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:25:53.519 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:25:53.519 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:53.520 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:53.521 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:53.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:53.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:53.521 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:53.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:53.522 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:53.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:53.522 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:53.534+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:53.523 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:53.621 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:53.635+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:53.658 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:53.658 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:53.667 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:53.721 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:53.735+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:53.792 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:53.792 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:53.801 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:53.822 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:53.837+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:53.923 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:53.936+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:53.966 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:53.967 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:53.975 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:54.024 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.037+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:54.088 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:25:54.107 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.107 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.116 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:54.125 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.139+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:54.225 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:54.239+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:54.240 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.240 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.249 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:54.333 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:54.347+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:54.413 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.414 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.422 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:54.466 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.480+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:54.528 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.541+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:54.551 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.552 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.560 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:54.643 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.657+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:54.725 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.725 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.729 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:54.743+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:54.734 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:54.830 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:54.845+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:54.899 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:54.899 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:54.908 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:54.931 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:54.945+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.032 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.045+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.033 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:55.034 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:55.043 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:55.080 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:25:55.132 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.146+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.167 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:55.167 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:55.176 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:25:55.233 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.247+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.300 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:55.300 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:55.309 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:25:55.334 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:55.349+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:55.434 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:55.435 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:55.435 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:55.449+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:55.444 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:25:55.523 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:25:55.536 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.550+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.569 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:55.569 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:55.578 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:25:55.661 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.675+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.676 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:55.696 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:25:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:25:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:55.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:55.738 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:25:55.752+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:25:55.746 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:25:55.746 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:25:55.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.250896888, 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:55.747 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:55.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:55.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:55.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:25:55.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:25:55.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:55.758 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.250784681, 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:55.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:55.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:55.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:25:55.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:25:55.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:25:55.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:25:55.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:55.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:56.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:25:56.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:25:56.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:56.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.176 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.25069444299999, 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:25:56.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:25:56.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:25:56.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:56.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:25:56.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:25:56.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:56.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.389 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.2505778, 2020-11-16T22:25:56.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:25:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:25:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:25:56.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:25:56.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:56.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:25:56.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:56.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:25:56.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:25:56.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:56.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.807 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.25042943099999, 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:25:56.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:56.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:56.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:56.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:25:56.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:25:57.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:57.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:25:57.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:25:57.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:25:57.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:25:57.021 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.25024006199999, 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:57.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:57.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:57.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:25:57.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:57.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:25:57.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:57.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:25:57.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:25:57.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:25:57.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:25:57.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:57.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:25:57.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:25:57.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:25:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.25002613999999, 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:57.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:57.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:25:57.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:25:57.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:57.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:25:57.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:25:57.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:57.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:25:57.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:25:57.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:57.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:25:57.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:25:57.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:25:57.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:57.856 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24982723900001, 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:57.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:25:57.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:57.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:25:57.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:57.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:25:57.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:57.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:25:58.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:25:58.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:25:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:58.068 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24964600199999, 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:25:58.069 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:58.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:25:58.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:25:58.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:25:58.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:25:58.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:25:58.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:25:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:58.282 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24901868800001, 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:58.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:58.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:58.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:58.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:25:58.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:58.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:25:58.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:58.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:25:58.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:25:58.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:58.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:25:58.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:25:58.697 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:58.698 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24872134500001, 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:58.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:25:58.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:25:58.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:25:58.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:58.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:25:58.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:25:58.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:58.910 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:58.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:58.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:58.911 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:25:58.911 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:59.114 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24831453300001, 2020-11-16T22:25:59.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:59.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:59.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.122 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24821531500001, 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:59.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:25:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:25:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:25:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:59.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:25:59.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:25:59.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:59.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:25:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.248020349, 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.542 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:25:59.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:25:59.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:25:59.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:25:59.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24745927699999, 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:25:59.552 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:25:59.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:25:59.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.559 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24730079599999, 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:25:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.24716788500001, 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:25:59.567 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:25:59.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:25:59.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 158.246999276, 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:25:59.575 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:25:59.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:25:59.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:25:59.581 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:59.582 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.011295515, 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:59.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:25:59.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:25:59.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:25:59.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:25:59.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:25:59.593 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.01116848800001, 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:25:59.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:25:59.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:25:59.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:25:59.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:25:59.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:25:59.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:25:59.604 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.605 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.605 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.605 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:25:59.605 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:25:59.605 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:25:59.606 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:25:59.607 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:25:59.608 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:25:59.609 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:59.620+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:59.707 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:59.721+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:59.745 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:25:59.745 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:25:59.746 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:25:59.746 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:25:59.755 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:25:59.769 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:25:59.789 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:25:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:25:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:25:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:25:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:25:59.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:25:59.793 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:25:59.838 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:59.852+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:59.908 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:25:59.922+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:25:59.919 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:25:59.919 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:25:59.928 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:00.012 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.026+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:00.085 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:26:00.093 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.093 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.103 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:00.146 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:00.160+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:00.211 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.225+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:00.239 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.239 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.249 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:00.332 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:00.346+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:00.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~102s 2020-11-16T22:26:00.413 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.427+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:00.414 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.414 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.423 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:00.514 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:00.527+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:00.588 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.588 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.597 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:00.641 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.655+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:00.715 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:00.729+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:00.724 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.724 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.733 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:00.816 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.830+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:00.897 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:00.897 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:00.906 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:00.947 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:00.961+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:01.018 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.032+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.070 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.070 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.079 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:01.119 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:01.133+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:01.208 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.209 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.218 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:01.220 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.233+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.321 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:01.336+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:01.341 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.342 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.351 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:01.422 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.436+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.476 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.476 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.485 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:01.529 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:01.543+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:01.612 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.613 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.621 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:01.624 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.638+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.725 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.739+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.785 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.785 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.793 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:01.795 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:01.825 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:01.839+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:01.920 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:01.920 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:01.926 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:01.940+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:01.929 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:01.946 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:01.965 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:01.966 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:01.966 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:01.966 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:01.966 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:01.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:01.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:02.027 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:02.040+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:02.056 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:02.057 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:02.066 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:02.096 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:26:02.127 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:02.142+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:02.228 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:02.242+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:02.230 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:02.230 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:02.231 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:02.231 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:02.231 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:02.239 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:02.328 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:02.342+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:02.365 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.87054515200001, 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.870432945, 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.87034270699999, 2020-11-16T22:26:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.87022606400001, 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.87007769499999, 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86988832599999, 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:02.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:02.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:02.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:02.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:02.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.636 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86967440399999, 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:02.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:02.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:02.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:02.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:02.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:02.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:02.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:02.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:02.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:02.848 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86947550299999, 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:02.849 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:02.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:03.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.061 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.869294266, 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:03.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:03.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:03.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86866695200001, 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.072 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:03.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:03.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:03.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:03.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86836960900001, 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.082 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:03.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:03.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:03.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:03.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:03.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:03.295 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86796279699999, 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:03.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:03.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.303 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86786357899999, 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:03.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:03.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:03.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.514 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86766861300001, 2020-11-16T22:26:03.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:03.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:03.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.524 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.867107541, 2020-11-16T22:26:03.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:03.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:03.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86694906, 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:03.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:03.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:03.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:03.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.743 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86681614899999, 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:03.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:03.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:03.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 164.86664754, 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:03.751 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:03.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:03.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:03.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:03.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:03.961 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 145.63094377900001, 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:03.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:03.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:03.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:03.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:03.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:04.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:04.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:04.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:04.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:04.177 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 145.63081675199999, 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:04.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:04.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:04.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:04.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:04.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:04.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:04.189 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:04.190 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:04.192 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:26:04.193 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:04.205+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:04.193 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:04.292 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:04.306+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:04.330 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:04.330 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:04.339 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:04.383 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:04.392 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:04.406+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:04.402 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:04.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:04.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:04.403 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:04.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:04.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:04.405 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:04.493 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:04.507+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:04.504 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:04.505 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:04.514 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:04.597 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:04.611+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:04.678 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:04.679 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:04.687 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:04.731 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:04.745+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:04.796 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:04.810+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:04.826 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:04.826 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:04.835 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:04.919 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:04.933+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:04.998 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.012+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.000 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.001 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.010 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:05.099 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.112+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.175 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.175 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.184 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:05.228 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:05.241+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:05.300 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:05.313+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:05.312 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.312 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.321 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:05.401 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.415+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.485 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.485 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.494 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:05.502 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.516+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.603 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.617+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.618 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.619 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.628 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:05.704 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:05.718+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:05.755 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.755 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.764 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:05.805 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:05.819+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:05.906 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:05.919+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:05.928 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:05.929 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:05.938 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:06.006 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:06.020+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:06.063 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:06.063 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:06.072 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:06.116 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:06.129+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:06.193 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:26:06.199 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:06.199 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:06.208 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:06.221+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:06.208 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:06.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~108s 2020-11-16T22:26:06.309 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:06.322+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:06.373 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:06.373 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:06.382 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:06.406 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:06.423 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:06.436+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:06.511 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:06.524+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:06.546 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:06.547 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:06.548 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:10.720 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:10.720 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:10.721 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:10.721 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:10.721 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:10.721 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:10.721 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:10.722 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:26:10.722 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:10.723 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:10.732+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:10.820 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:10.834+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:10.860 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:10.861 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:10.861 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:10.861 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:10.870 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:10.953 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:10.967+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:11.022 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:11.035+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:11.035 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:11.035 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:11.035 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:11.035 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:11.035 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:11.043 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:11.126 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:11.140+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:11.209 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:11.209 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71422976299999, 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:11.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:11.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71411755599999, 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.222 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:11.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:11.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71402731800001, 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:11.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:11.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:11.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:11.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:11.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:11.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:11.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:11.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:11.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71391067499999, 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:11.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:11.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:11.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:11.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:11.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:11.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71376230600001, 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.659 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:11.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:11.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:11.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:11.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:11.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:11.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71357293700001, 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:11.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:11.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:11.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:11.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:12.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:12.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.084 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71335901500001, 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:12.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:12.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:12.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:12.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:12.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.713160114, 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.096 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:12.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:12.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:12.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.106 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.106 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:12.106 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71297887700001, 2020-11-16T22:26:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.712351563, 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:12.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:12.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:12.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71205422, 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.532 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:12.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:12.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:12.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:12.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:12.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.743 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.711647408, 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:12.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:12.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:12.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71154819, 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:12.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:12.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:12.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:12.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:12.967 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71135322399999, 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:12.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:12.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:12.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:12.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:12.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:12.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:12.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:12.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:13.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:13.178 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.178 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.178 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71079215200001, 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.179 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:13.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:13.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.186 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71063367100001, 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:13.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:13.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:13.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71050076, 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.194 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:13.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:13.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:13.200 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:13.201 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.71033215099999, 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:13.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:13.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:13.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:13.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.411 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 154.47462838999999, 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:13.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:13.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:13.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:13.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:13.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 154.474501363, 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:13.423 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:13.424 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:13.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:13.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:13.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:13.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:13.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:13.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:13.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:13.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:13.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:13.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:13.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:13.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:13.636 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:13.637 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:13.638 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:26:13.639 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:13.651+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:13.640 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:13.738 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:13.751+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:13.777 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:13.777 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:13.777 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:13.778 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:13.786 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:13.796 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:13.817 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:13.817 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:13.817 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:13.817 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:13.818 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:13.819 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:13.819 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:13.840 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:13.854+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:13.940 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:13.955+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:13.951 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:13.951 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:13.960 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:14.041 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.055+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.084 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.085 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.094 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:14.141 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:14.155+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:14.230 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.230 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.240 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:14.242 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.255+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.342 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.356+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.364 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.364 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.373 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:14.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~116s 2020-11-16T22:26:14.443 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:14.457+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:14.537 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.538 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.544 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.558+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.547 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:14.644 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.659+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.672 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.673 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.682 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:14.745 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:14.758+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:14.805 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.806 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.814 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:14.856 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:14.870+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:26:14.946 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:14.960+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:14.980 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:14.980 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:14.989 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:15.047 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.061+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.115 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.115 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.124 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:15.167 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:15.181+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:15.249 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.262+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.250 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.250 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.259 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:15.349 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:15.363+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:15.425 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.425 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.435 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:15.479 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.493+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.551 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.566+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.561 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.561 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.571 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:15.653 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:15.667+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:15.736 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.736 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.745 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:15.789 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.803+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.820 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:15.854 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.868+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.870 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:15.870 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:15.879 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:15.955 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:15.969+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:15.976 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:15.995 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:15.998 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:15.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:16.008 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:16.009 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:16.009 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:16.009 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:16.018 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:16.056 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:16.070+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:16.142 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:16.142 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:16.142 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:16.142 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:16.143 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:16.151 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:16.156 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:16.170+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:16.257 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:16.271+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:16.277 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:16.277 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:16.278 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.78197609700001, 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:16.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.78186389000001, 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.290 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:16.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:16.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:16.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.781773652, 2020-11-16T22:26:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:16.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:16.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:16.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:16.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.312 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.78165700900001, 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:16.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:16.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:16.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:16.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:16.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.78150864, 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.324 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:16.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:16.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:16.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:16.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:16.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:16.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.781319271, 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:16.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:16.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:16.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:16.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:16.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.346 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.781105349, 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:16.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:16.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:16.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:16.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.780906448, 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.358 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:16.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.780725211, 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:16.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:16.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:16.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.78009789699999, 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:16.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:16.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:16.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:16.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.387 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.77980055399999, 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:16.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:16.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:16.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:16.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:16.397 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.779393742, 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.77929452399999, 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.406 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:16.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:16.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:16.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.415 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.77909955800001, 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:16.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:16.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:16.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:16.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.425 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.778538486, 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:16.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:16.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:16.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.778380005, 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:16.433 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:16.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:16.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.439 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.77824709399999, 2020-11-16T22:26:16.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:16.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.447 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.77807848500001, 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:16.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 159.54237472400001, 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.455 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:16.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:16.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:16.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.465 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 159.54224769699999, 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:16.466 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:16.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:16.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:16.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:16.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:16.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:16.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.476 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.477 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.477 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:16.477 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:16.477 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:16.478 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:16.492+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:16.479 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:16.579 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:16.593+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:16.605 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:16.606 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:16.606 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:16.606 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:16.615 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:16.679 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:16.693+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:16.738 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:16.739 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:16.739 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:16.739 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:16.739 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:16.747 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:16.791 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:16.805+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:16.873 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:16.873 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:16.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 186.797697669, 2020-11-16T22:26:16.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:16.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:16.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:16.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 186.686267988, 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.377831437, 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:16.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.900 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37771613699999, 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:16.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:16.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:16.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:16.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.910 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.377607377, 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:16.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:16.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:16.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:16.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.920 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37752502800001, 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:16.921 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:16.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.930 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37743721999999, 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:16.931 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:16.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:16.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:16.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:16.941 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37734268099999, 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:16.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:16.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:16.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.951 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37725649500001, 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:16.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:16.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:16.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:16.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:16.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:16.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:16.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:16.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:16.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37715610999999, 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.971 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:16.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:16.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:16.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:16.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37204278900001, 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:16.981 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:16.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:16.988 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.371936616, 2020-11-16T22:26:16.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:16.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:16.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:16.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:16.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:16.997 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.371762606, 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:16.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:16.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:17.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:17.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:17.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:17.005 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37166632700001, 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:17.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:17.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:17.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:17.013 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37147862500001, 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.019 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37128667100001, 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:17.020 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:17.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:17.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:17.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.028 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37112410700001, 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:17.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:17.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37094824499999, 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.037 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:17.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:17.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:17.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.044 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37078020000001, 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.045 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:17.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:17.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:17.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:17.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 179.37053907000001, 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:17.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:17.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:17.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:17.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:17.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:17.263 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:17.264 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:17.264 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:26:17.266 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:17.279+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:17.266 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:17.366 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:17.380+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:17.402 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:17.402 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:17.403 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:17.403 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:17.413 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:17.467 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:17.481+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:17.537 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:17.538 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:17.538 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:17.538 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:17.538 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:17.546 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:17.567 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:17.581+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:17.668 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:17.682+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:17.673 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:17.673 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:17.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:17.673 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:17.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17786109400001, 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.674 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:17.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:17.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:17.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:17.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:17.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:17.684 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17774888700001, 2020-11-16T22:26:17.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:17.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:17.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:17.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:17.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17765864899999, 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.697 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:17.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:17.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:17.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:17.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.707 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17754200600001, 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:17.708 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:17.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:17.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:17.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:17.719 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17739363699999, 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:17.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:17.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:17.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:17.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.730 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.177204268, 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:17.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:17.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:17.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.176990346, 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.742 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:17.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:17.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:17.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.752 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17679144499999, 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.753 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:17.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.176610208, 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:17.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:17.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17598289399999, 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.773 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:17.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:17.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.782 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17568555099999, 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:17.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:17.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:17.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:17.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:17.792 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17527873899999, 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:17.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.800 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17517952099999, 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:17.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:17.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:17.810 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17498455500001, 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:17.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:17.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:17.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:17.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.174423483, 2020-11-16T22:26:17.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:17.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:17.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.827 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.174265002, 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:17.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:17.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.17413209099999, 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:17.835 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:17.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.173963482, 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:17.849 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 160.93825972100001, 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:17.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:17.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:17.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:17.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.860 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 160.93813269399999, 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:17.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:17.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:17.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:17.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:17.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:17.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:17.872 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:17.873 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:17.873 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:17.887+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:17.883 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:26:17.973 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:17.987+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:17.999 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:18.010 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:18.010 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:18.010 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:18.011 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:18.019 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:18.074 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.088+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.140 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:18.160 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:18.161 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:18.163 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:18.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:18.175 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.190+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.186 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.186 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.195 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:18.276 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:18.290+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:18.320 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.320 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.329 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:18.377 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.390+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.457 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.457 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.466 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:18.507 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.521+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.578 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.592+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.631 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.632 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.640 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:18.679 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:18.693+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:18.780 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.794+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.805 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.806 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.819 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:18.881 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:18.896+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:18.947 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:18.947 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:18.956 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:18.999 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.013+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.079 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.080 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.083 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.097+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.088 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:19.184 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:19.198+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:19.253 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.254 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.262 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:19.285 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.299+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.386 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.400+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.389 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.389 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.398 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:19.486 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.500+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.522 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.523 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.532 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:19.586 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.601+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.657 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.657 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.666 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:19.710 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.724+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.788 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:19.801+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:19.793 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.793 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.803 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:19.888 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:19.902+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:19.894 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:26:19.967 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:19.968 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:19.977 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:20.018 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.032+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.090 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.104+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.141 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:20.142 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:20.151 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:20.164 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:20.192 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.206+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~122s 2020-11-16T22:26:20.277 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:20.277 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:20.286 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:20.291 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.305+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.305 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:20.325 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:20.326 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:20.326 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:20.327 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:20.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:20.392 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.406+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.409 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:20.410 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:20.410 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:20.410 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:20.410 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:20.419 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:20.492 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:20.506+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:20.545 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:20.545 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.049830004, 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.546 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:20.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:20.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:20.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:20.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:20.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:20.557 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.049717797, 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:20.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:20.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:20.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:20.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:20.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.568 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04962755899999, 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:20.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:20.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.049510916, 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.580 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:20.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:20.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:20.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:20.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:20.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:20.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:20.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.795 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04936254699999, 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:20.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:20.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:20.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04917317799999, 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.807 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:20.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:20.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:20.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.817 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04895925599999, 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:20.818 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:20.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:20.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:20.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:20.829 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04876035500001, 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:20.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:20.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:20.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:20.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:20.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04857911799999, 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.843 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:20.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:20.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:20.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:20.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:20.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:20.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:20.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04795180400001, 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:20.853 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:20.854 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:20.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:20.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:20.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:21.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:21.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:21.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:21.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.066 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04765446100001, 2020-11-16T22:26:21.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:21.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:21.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:21.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.076 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04724764900001, 2020-11-16T22:26:21.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:21.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:21.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:21.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:21.085 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04714843100001, 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:21.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:21.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:21.095 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.046953465, 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:21.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:21.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:21.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.105 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04639239299999, 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:21.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:21.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:21.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04623391199999, 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:21.113 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:21.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:21.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:21.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:21.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.322 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.04610100100001, 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:21.323 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:21.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:21.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.330 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 183.045932392, 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 163.810228631, 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.338 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:21.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:21.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:21.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 163.81010160400001, 2020-11-16T22:26:21.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:21.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:21.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:21.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:21.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:21.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:21.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:21.360 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:26:21.362 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:21.375+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:21.363 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:21.462 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:21.476+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:21.498 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:21.499 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:21.499 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:21.508 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:21.564 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:21.577+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:21.631 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:21.631 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:21.632 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:21.632 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:21.632 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:21.640 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:21.684 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:21.698+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:21.765 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:21.777+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:21.767 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:21.767 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:21.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 191.69169852300001, 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:21.768 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:21.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:21.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:21.775 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 191.58026884200001, 2020-11-16T22:26:21.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:21.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:21.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:21.783 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.27183229100001, 2020-11-16T22:26:21.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:21.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:21.793 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:21.794 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.27171699100001, 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:21.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:21.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:21.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:21.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:21.803 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:21.804 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.27160823099999, 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:21.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:21.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:21.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:21.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:21.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.27152588199999, 2020-11-16T22:26:21.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:21.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:21.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:21.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:21.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:21.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:21.824 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.271438074, 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:21.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:21.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:21.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:21.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:21.833 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.271343535, 2020-11-16T22:26:21.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:21.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:21.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:21.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:21.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:21.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:21.844 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.271257349, 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:21.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:21.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:21.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:21.854 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.271156964, 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:21.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:21.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:21.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:21.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:21.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:21.865 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:22.066 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:22.066 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26604364299999, 2020-11-16T22:26:22.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:22.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:22.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:22.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:22.074 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26593747000001, 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:22.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:22.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:22.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:22.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:22.082 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26576345999999, 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:22.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:22.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:22.089 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:22.090 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.265667181, 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:22.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:22.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:22.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:22.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:22.098 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26547947899999, 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:22.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:22.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26528752499999, 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.105 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:22.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:22.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:22.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.112 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.265124961, 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.113 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26494909900001, 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.121 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:22.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:22.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.264781054, 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.332 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:22.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.26453992399999, 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:22.340 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:22.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:22.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:22.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:22.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:22.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:22.347 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:22.348 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:22.348 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:22.350 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:22.363+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:22.350 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:22.449 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:22.462+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:22.487 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:22.488 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:22.497 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:22.500 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:22.520 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:22.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:22.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:22.521 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:22.521 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:22.523 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:22.523 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:22.580 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:22.594+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:22.651 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:22.665+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:22.662 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:22.662 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:22.671 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:22.752 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:22.766+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:22.835 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:22.835 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:22.844 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:22.892 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:22.906+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:22.953 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:22.967+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:22.980 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:22.981 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:22.990 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:23.073 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.087+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.154 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.154 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.155 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.169+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.163 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:23.255 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:23.269+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:23.328 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.328 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.337 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:23.363 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:26:23.380 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:23.394+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:23.457 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.471+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.464 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.464 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.473 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:23.558 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:23.572+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:23.637 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.638 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.647 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:23.659 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.673+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.759 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.773+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.812 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.812 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.821 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:23.860 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:23.874+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:23.912 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:26:23.953 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:23.953 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:23.960 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:23.975+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:23.962 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:24.061 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:24.075+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:24.127 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:24.127 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:24.136 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:24.177 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:24.191+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:24.263 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:24.277+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:24.301 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:24.301 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:24.310 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:24.364 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:24.378+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:24.438 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:24.438 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:24.447 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:24.488 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:24.502+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:24.524 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:24.565 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:24.580+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:24.611 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:24.612 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:24.621 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:24.666 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:24.680+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:24.667 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:24.686 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:24.689 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:24.690 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:24.747 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25206419700001, 2020-11-16T22:26:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25195199000001, 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.251861752, 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25174510900001, 2020-11-16T22:26:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:24.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:24.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:24.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:24.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:24.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:24.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:24.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:24.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:24.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25159674, 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:25.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:25.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:25.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:25.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:25.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:25.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:25.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:25.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:25.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.251407371, 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:25.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:25.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:25.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:25.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:25.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:25.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:25.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:25.630 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:25.631 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.251193449, 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:25.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:25.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:25.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:25.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:25.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:25.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:25.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:25.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:26.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:26.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:26.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.048 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25099454799999, 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:26.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:26.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:26.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:26.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:26.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:26.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:26.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.260 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.250813311, 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:26.261 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:26.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:26.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:26.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:26.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.473 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.25018599699999, 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:26.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:26.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:26.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:26.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:26.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:26.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:26.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:26.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:26.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.687 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:26.688 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:26.688 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24988865399999, 2020-11-16T22:26:26.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:26.890 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:26.890 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:26.890 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:26.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:26.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:26.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:26.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:26.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:26.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:26.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:26.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:26.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.899 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:26.900 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:26.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:26.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:26.900 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24948184199999, 2020-11-16T22:26:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:27.102 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:27.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:27.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:27.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:27.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:27.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:27.110 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24938262399999, 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:27.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:27.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:27.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:27.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:27.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:27.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:27.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24918765800001, 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:27.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:27.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:27.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:27.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:27.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:27.738 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.248626586, 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:27.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:27.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:27.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:27.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:27.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:27.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:27.948 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.248468105, 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:27.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:27.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:27.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:27.951 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:27.951 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:28.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:28.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:28.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:28.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:28.158 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24833519399999, 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:28.159 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:28.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:28.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:28.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:28.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 187.24816658500001, 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:28.369 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:28.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:28.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:28.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:28.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.578 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:28.578 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 168.01246282400001, 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:28.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:28.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:28.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:28.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:28.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:28.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:28.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:28.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:28.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:28.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 168.01233579699999, 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:28.794 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:28.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:28.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:28.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:28.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:28.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:28.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:28.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:29.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:29.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:29.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:29.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:29.007 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:29.008 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:26:29.009 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:29.211 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:29.022+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:29.212 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:29.214 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.227+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:29.315 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.328+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:29.349 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:29.349 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:29.358 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:29.370 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:29.389 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:29.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:29.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:29.415 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:29.429+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:29.516 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.530+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:29.523 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:29.523 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:29.532 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:29.618 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.632+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:29.697 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:29.697 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:29.706 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:29.749 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:29.763+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:29.819 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.833+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:29.842 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:29.843 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:29.852 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:29.918 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:26:29.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~131s 2020-11-16T22:26:29.920 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:29.934+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:30.017 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.018 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.021 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.035+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.027 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:30.121 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.135+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.192 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.192 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.201 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:30.242 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.256+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.322 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.336+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.327 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.327 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.336 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:30.423 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.437+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.460 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.460 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.469 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:30.524 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:30.538+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:30.595 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.595 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.604 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:30.645 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:30.659+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:30.726 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.740+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.729 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.729 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.739 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:30.826 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.840+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:30.903 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:30.904 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:30.915 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:30.956 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:30.970+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:31.029 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:31.043+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:31.080 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:31.080 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:31.089 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:31.133 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:31.147+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:31.217 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:31.217 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:31.227 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:31.268 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:31.282+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:31.330 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:31.344+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:31.391 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:31.391 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:31.394 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:31.400 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:31.444 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:31.458+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:31.525 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:31.526 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:31.531 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:31.544+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:31.534 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:31.545 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:31.565 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:31.566 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:31.566 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:31.566 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:31.566 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:31.568 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:31.569 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:31.631 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:31.645+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:31.659 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:31.660 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:31.660 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:31.660 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:31.668 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:31.732 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:31.746+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:31.832 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:31.832 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:31.833 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:31.833 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:31.833 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:31.833 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:31.846+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:31.842 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:31.929 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:26:31.934 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:31.948+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:32.008 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:32.008 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51306041300001, 2020-11-16T22:26:32.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:32.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:32.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:32.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:32.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:32.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.020 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.512948206, 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:32.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:32.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:32.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:32.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.031 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51285796799999, 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:32.032 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:32.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.043 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51274132500001, 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:32.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:32.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51259295599999, 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.055 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:32.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:32.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:32.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:32.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51240358699999, 2020-11-16T22:26:32.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:32.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:32.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:32.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:32.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:32.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.077 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51218966499999, 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:32.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:32.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:32.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:32.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:32.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.088 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51199076399999, 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:32.089 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:32.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.098 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.511809527, 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:32.099 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:32.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:32.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.108 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.109 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.109 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:32.310 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51118221300001, 2020-11-16T22:26:32.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:32.310 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.310 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51088487000001, 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.329 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51047805799999, 2020-11-16T22:26:32.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:32.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:32.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:32.337 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51037883999999, 2020-11-16T22:26:32.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:32.339 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.542 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.542 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:32.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:32.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:32.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.551 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.51018387400001, 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:32.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:32.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:32.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:32.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.561 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.509622802, 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:32.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:32.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:32.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.509464321, 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.569 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:32.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:32.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:32.575 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:32.576 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.50933140999999, 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:32.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:32.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:32.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.583 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 194.509162801, 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:32.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:32.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:32.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.590 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 175.27345904000001, 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:32.591 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:32.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:32.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:32.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:32.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:32.602 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 175.27333201299999, 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:32.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:32.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:32.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:32.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:32.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:32.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:32.613 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:32.615 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:32.616 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:32.628+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:32.716 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:32.730+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:32.751 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:32.751 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:32.751 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:32.752 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:32.760 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:32.816 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:32.830+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:32.884 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:32.884 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:32.884 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:32.885 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:32.885 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:32.893 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:26:32.936 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:32.950+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:33.017 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:33.031+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:33.021 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:33.021 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:33.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.945560188, 2020-11-16T22:26:33.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:33.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:33.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:33.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:33.030 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.834130507, 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:33.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:33.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:33.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.525693956, 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:33.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:33.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.048 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52557865599999, 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:33.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:33.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:33.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:33.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.058 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.525469896, 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:33.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:33.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:33.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:33.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.068 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52538754700001, 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:33.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:33.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:33.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.078 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52529973899999, 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:33.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:33.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:33.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:33.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.088 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52520519999999, 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:33.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:33.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:33.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:33.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.098 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52511901400001, 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:33.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:33.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:33.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.52501862899999, 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51990530800001, 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:33.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:33.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.126 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.519799135, 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:33.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:33.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:33.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.519625125, 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:33.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.142 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51952884599999, 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:33.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:33.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:33.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.150 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51934114400001, 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:33.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:33.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:33.156 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:33.157 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51914919000001, 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:33.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:33.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:33.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:33.164 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:33.165 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51898662599999, 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:33.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:33.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:33.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:33.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51881076399999, 2020-11-16T22:26:33.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:33.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:33.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:33.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:33.180 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51864271900001, 2020-11-16T22:26:33.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:33.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:33.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:33.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:33.188 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.51840158900001, 2020-11-16T22:26:33.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:33.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:33.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:33.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:33.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.196 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.197 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.197 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:33.197 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:33.197 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:33.198 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:33.212+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:33.199 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:33.299 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:33.313+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:33.336 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:33.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:33.337 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:33.337 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:33.346 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:33.429 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:33.443+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:33.500 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:33.514+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:33.511 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:33.512 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:33.512 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:33.512 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:33.512 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:33.521 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:33.570 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:33.600 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:33.615+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:33.689 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:33.689 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:33.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19382755199999, 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:33.690 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:33.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:33.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:33.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.701 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19371534499999, 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:33.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:33.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:33.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:33.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:33.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.193625107, 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.713 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:33.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:33.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:33.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:33.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.724 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19350846399999, 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:33.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:33.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:33.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:33.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:33.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.193360095, 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.736 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:33.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:33.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:33.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:33.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:33.746 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:33.747 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19317072600001, 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:33.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:33.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:33.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:33.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:33.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.758 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.192956804, 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:33.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:33.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:33.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:33.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:33.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.192757903, 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.770 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:33.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:33.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:33.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:33.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19257666600001, 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.780 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:33.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:33.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:33.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:33.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19194935199999, 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.790 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:33.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:33.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:33.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:33.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19165200899999, 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.800 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:33.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:33.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:33.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.191245197, 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.810 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:33.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:33.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:33.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.191145979, 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.818 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:33.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:33.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19095101299999, 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.828 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:33.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:33.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19038994100001, 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:33.838 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:33.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:33.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.844 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.19023146000001, 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:33.845 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.190098549, 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.18992994000001, 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 176.95422617899999, 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 176.954099152, 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:33.889 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:33.892 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:33.892 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:33.892 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:33.893 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:33.904+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:33.992 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.006+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.030 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.030 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.040 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:34.124 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.138+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.193 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.207+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.205 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.205 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.214 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:34.297 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.311+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.378 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.378 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.387 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:34.431 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:34.445+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:34.496 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:34.510+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:34.518 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.518 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.527 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:34.596 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.610+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.692 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.693 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.697 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:34.710+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:34.702 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:34.797 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:34.811+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:34.866 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:34.866 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:34.875 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:34.919 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:34.933+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:34.939 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:26:34.998 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.012+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.004 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.004 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.013 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:35.099 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.113+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.178 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.178 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.188 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:35.200 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.214+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.300 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.314+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.312 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.312 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.321 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:35.401 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:35.416+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:35.449 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.449 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.458 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:35.503 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.516+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.583 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.583 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.591 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:35.604 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:35.618+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:35.704 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:35.718+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:35.723 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~137s 2020-11-16T22:26:35.756 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.756 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.765 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:35.809 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:35.823+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:35.893 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:35.894 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:35.894 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:35.903 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:35.939 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:26:35.944 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:35.958+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.008 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.022+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.047 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:36.067 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:36.067 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:36.067 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:36.068 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:36.068 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:36.069 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:36.069 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:36.069 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:36.069 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:36.069 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:36.070 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:36.070 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:36.078 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:36.119 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.133+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.210 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:36.224+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:36.243 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:36.244 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 206.16788749200001, 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:36.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:36.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:36.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 206.056457811, 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.253 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:36.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:36.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:36.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.260 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74802126, 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:36.261 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:36.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:36.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.270 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74790596, 2020-11-16T22:26:36.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:36.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:36.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:36.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:36.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.282 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74779720000001, 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:36.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:36.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:36.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:36.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.292 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74771485100001, 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:36.293 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.302 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74762704299999, 2020-11-16T22:26:36.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:36.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:36.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.312 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74753250399999, 2020-11-16T22:26:36.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:36.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:36.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.322 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74744631799999, 2020-11-16T22:26:36.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:36.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:36.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:36.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:36.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74734593299999, 2020-11-16T22:26:36.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:36.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:36.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:36.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:36.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:36.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:36.343 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74223261200001, 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:36.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.742126439, 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74195242900001, 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:36.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:36.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.367 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74185614999999, 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:36.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:36.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:36.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74166844800001, 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:36.382 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74147649400001, 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:36.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:36.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74131392999999, 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.391 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:36.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:36.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.741138068, 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.399 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:36.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:36.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:36.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74097002299999, 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:36.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:36.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:36.415 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 198.74072889300001, 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:36.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:36.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:36.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:36.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:36.423 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:36.424 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.439+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.425 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:36.525 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.539+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.555 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:36.556 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:36.556 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:36.564 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:36.625 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.639+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.689 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:36.689 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:36.689 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:36.689 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:36.689 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:36.698 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:36.726 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:36.740+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:36.825 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:36.825 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:36.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.329872456, 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:36.826 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:36.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:36.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:36.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:36.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.836 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.837 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.329760249, 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:36.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:36.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:36.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:36.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:36.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.848 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32967001099999, 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.849 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:36.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:36.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:36.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:36.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:36.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:36.860 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32955336800001, 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:36.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:36.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:36.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:36.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:36.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.871 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32940499899999, 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:36.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:36.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:36.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:36.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32921562999999, 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.883 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:36.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:36.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:36.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:36.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32900170799999, 2020-11-16T22:26:36.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:36.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:36.905 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32880280699999, 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:36.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:36.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:36.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:36.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:36.915 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32862157, 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:36.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:36.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:36.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:36.924 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:36.925 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32799425600001, 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:36.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:36.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:36.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:36.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:36.934 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32769691300001, 2020-11-16T22:26:36.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:36.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:36.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.944 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32729010099999, 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:36.945 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:36.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:36.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.952 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32719088299999, 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.953 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:36.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:36.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:36.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:36.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:36.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.326995917, 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:36.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:36.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:36.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.977 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32643484499999, 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:36.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:36.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:36.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.984 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32627636399999, 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.985 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:36.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:36.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:36.991 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:36.992 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.32614345299999, 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:36.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:36.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:36.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:36.999 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.325974844, 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:37.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:37.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:37.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:37.006 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.090271083, 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:37.007 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:37.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:37.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:37.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:37.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:37.018 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 180.09014405600001, 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:37.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:37.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:37.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:37.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:37.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:37.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:37.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:37.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:37.029 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:37.030 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:37.031 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:37.045+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:37.031 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:37.131 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:37.146+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:37.168 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:37.168 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:37.168 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:37.169 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:37.178 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:37.233 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:37.247+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:37.333 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:37.347+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:37.342 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:37.343 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:37.343 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:37.343 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:37.343 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:37.353 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:37.434 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:37.448+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.403534275, 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.29210459399999, 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.488 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:37.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:37.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:37.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98366804299999, 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:37.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:37.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:37.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:37.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98355274299999, 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.506 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:37.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:37.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:37.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.983443983, 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.516 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:37.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:37.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.983361634, 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:37.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:37.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:37.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:37.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98327382599999, 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:37.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:37.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:37.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:37.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.545 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98317928700001, 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:37.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:37.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:37.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.555 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98309310100001, 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:37.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:37.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:37.564 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:37.565 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.98299271600001, 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:37.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:37.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:37.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:37.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:37.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.574 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.977879395, 2020-11-16T22:26:37.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.582 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.977773222, 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:37.583 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:37.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.590 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.977599212, 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:37.591 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:37.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:37.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:37.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.97750293300001, 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.599 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:37.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:37.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:37.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.97731523100001, 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.607 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:37.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.977123277, 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:37.621 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.97696071300001, 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:37.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:37.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:37.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:37.629 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.97678485099999, 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:37.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:37.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:37.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:37.636 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.97661680600001, 2020-11-16T22:26:37.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:37.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:37.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:37.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.644 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.976375676, 2020-11-16T22:26:37.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:37.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:37.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:37.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:37.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:37.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.652 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:37.653 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:37.653 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:37.654 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:37.668+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:37.655 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:37.755 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:37.768+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:37.787 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:37.787 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:37.796 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:37.855 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:37.869+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:37.952 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:26:37.955 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:37.970+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:37.961 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:37.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:37.962 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:37.962 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:37.962 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:37.971 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:38.057 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:38.071+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:38.071 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:38.097 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:38.097 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:38.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:38.097 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:38.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.60146096099999, 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.098 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:38.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:38.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:38.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.60134875400001, 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:38.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:38.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:38.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.601258516, 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:38.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:38.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:38.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:38.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.60114187299999, 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.132 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:38.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:38.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:38.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:38.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:38.142 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.600993504, 2020-11-16T22:26:38.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:38.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:38.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:38.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:38.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.154 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.600804135, 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:38.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:38.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:38.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:38.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.600590213, 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.600391312, 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.60021007500001, 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59958276099999, 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59928541799999, 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.598878606, 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.598779388, 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59858442199999, 2020-11-16T22:26:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59802335000001, 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59786486900001, 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.597731958, 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 200.59756334900001, 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 181.36185958799999, 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 181.361732561, 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:38.297 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:38.299 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:38.299 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:38.300 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:38.300 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:38.300 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:38.300 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:38.300 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:38.301 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:38.301 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:38.302 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:38.313+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:38.400 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:38.414+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:38.433 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:38.433 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:38.443 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:38.501 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:38.515+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:38.602 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:38.617+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:38.607 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:38.608 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:38.617 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:38.703 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:38.717+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:38.781 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:38.782 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:38.790 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:38.834 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:38.848+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:38.903 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:38.917+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:38.922 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:38.922 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:38.931 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:26:39.014 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.028+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.096 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.097 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.105 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.119+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.106 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:39.205 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.219+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.270 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.270 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.279 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:39.323 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:39.337+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:39.406 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.406 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.407 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.420+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.415 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:39.507 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:39.521+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:39.579 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.580 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.589 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:39.608 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.622+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.710 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:39.724+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:39.754 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.754 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.763 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:39.811 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.825+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:39.891 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:39.891 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:39.899 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:39.940 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:39.954+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:40.013 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.027+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.064 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:40.064 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:40.073 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:40.114 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.128+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.197 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:40.197 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:40.206 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:40.249 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.263+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.302 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:40.315 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:40.329+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:40.334 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:40.334 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:40.343 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:40.426 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.440+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.455 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:40.474 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:40.474 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:40.475 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:40.475 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:40.475 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:40.477 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:40.477 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:40.508 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:40.509 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:40.509 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:40.509 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:40.509 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:40.517 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:40.518 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.531+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.618 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:40.632+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.684 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 210.608184661, 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:40.685 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:40.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:40.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:40.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:40.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:40.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:40.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:40.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.693 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 210.49675497999999, 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:40.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:40.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:40.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18831842899999, 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:40.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:40.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:40.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:40.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:40.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:40.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:40.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:40.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18820312899999, 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:40.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:40.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:40.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:40.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:41.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:41.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:41.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:41.331 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.188094369, 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:41.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:41.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:41.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:41.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:41.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:41.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:41.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:41.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.543 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18801202, 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:41.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:41.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:41.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:41.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:41.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:41.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:41.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:41.756 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18792421200001, 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:41.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18782967300001, 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:42.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:42.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:42.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:42.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:42.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:42.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.182 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18774348700001, 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:42.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:42.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:42.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:42.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:42.598 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18764310200001, 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:42.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:42.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:42.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:42.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:42.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:42.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:42.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:42.810 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.182529781, 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:42.811 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:42.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:42.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:42.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:42.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:42.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18242360799999, 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.182249598, 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:43.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:43.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:43.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:43.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.443 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:43.646 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:43.646 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18215331900001, 2020-11-16T22:26:43.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:43.646 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:43.646 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:43.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:43.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:43.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.653 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.181965617, 2020-11-16T22:26:43.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:43.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:43.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:43.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:43.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:43.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:43.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:43.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:44.066 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.066 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.066 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.181773663, 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:44.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:44.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:44.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:44.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:44.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18161109900001, 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18143523699999, 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.18126719200001, 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:44.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:44.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:44.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:44.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 203.181026062, 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:44.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:44.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:44.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:44.513 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.514 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:44.514 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:44.514 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:44.515 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:26:44.516 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:44.517 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:44.529+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:44.518 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:44.519 DEBUG:teuthology.orchestra.console:expect before: b'\r\n[Enter `^Ec?\' for help]\r\n\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H\x1b[0m\x1b[2;37;40mInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent GE v1.5.13 \x1b[03;00HPXE 2.1 Build 092 (WfM 2.0) \x1b[04;00HPress Ctrl+S to enter the Setup Menu. \x1b[05;00H \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[04;39H\x1b[01;00H \x1b[04;00HPress Ctrl+S to enter the Setup Menu.. \x1b[04;39H\x1b[04;00H \x1b[05;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H \x1b[03;00HIntel(R) Boot Agent XE v2.1.60 \x1b[04;00HCopyright (C) 1997-2009, Intel Corporation \x1b[05;00H \x1b[06;00HCLIENT MAC ADDR: 0C C4 7A 8F D5 15 GUID: 00000000 0000 0000 0000 0CC47AD8D9E4 \x1b[07;00HDHCP.- \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[07;06H\x1b[01;00H \x1b[07;00HDHCP.| \x1b[07;06H\x1b[07;00HDHCP./ \x1b[07;06H\x1b[07;00HDHCP.- \x1b[07;06H\x1b[07;00HDHCP.\\ \x1b[07;06H\x1b[07;00HDHCP.| \x1b[07;06H\x1b[07;00HDHCP./ \x1b[07;06H\x1b[07;00HDHCP.- \x1b[08;00HGATEWAY IP: 172.21.15.254 \x1b[09;00HPXE->EB: !PXE at 96D3:0070, entry point at 96D3:0106 \x1b[10;00H UNDI code segment 96D3:3E70, data segment 8F94:73F0 (574-619kB) \x1b[11;00H UNDI device is PCI 03:00.1, type DIX+802.3 \x1b[12;00H 574kB free base memory after PXE unload \x1b[13;00HiPXE initialising devices... \x1b[13;29H\x1b[07;00HCLIENT IP: 172.21.15.122 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 \x1b[13;29H\x1b[13;00HiPXE initialising devices...ok \x1b[17;00H\x1b[0m\x1b[1;37;40miPXE 1.0.0+ (1b67a)\x1b[0m\x1b[2;37;40m -- Open Source Network Boot Firmware -- \x1b[0m\x1b[2;36;40mhttp://ipxe.org\x1b[0m\x1b[2;37;40m \x1b[18;00HFeatures: DNS HTTP HTTPS iSCSI TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT \x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)... \x1b[19;40H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15).... \x1b[19;41H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)..... \x1b[19;42H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)...... ok \x1b[20;00HReceived DHCP answer on interface net0 \x1b[H\x1b[J\x1b[1;1H\x1b[?25l\x1b[m\x1b[H\x1b[J\x1b[1;1H\x1b[20;7H\x1b[mUse the ^ and v keys to change the selection. \n\r Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[4;80H \x1b[7m\x1b[4;1H CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[m\x1b[4;79H\x1b[m\x1b[m\x1b[5;1H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[m\x1b[5;79H\x1b[m\x1b[m\x1b[6;1H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[m\x1b[6;79H\x1b[m\x1b[m\x1b[7;1H \x1b[m\x1b[7;79H\x1b[m\x1b[m\x1b[8;1H \x1b[m\x1b[8;79H\x1b[m\x1b[m\x1b[9;1H \x1b[m\x1b[9;79H\x1b[m\x1b[m\x1b[10;1H \x1b[m\x1b[10;79H\x1b[m\x1b[m\x1b[11;1H \x1b[m\x1b[11;79H\x1b[m\x1b[m\x1b[12;1H \x1b[m\x1b[12;79H\x1b[m\x1b[m\x1b[13;1H \x1b[m\x1b[13;79H\x1b[m\x1b[m\x1b[14;1H \x1b[m\x1b[14;79H\x1b[m\x1b[m\x1b[15;1H \x1b[m\x1b[15;79H\x1b[m\x1b[m\x1b[16;1H \x1b[m\x1b[16;79H\x1b[m\x1b[m\x1b[17;1H \x1b[m\x1b[17;79H\x1b[m\x1b[m\x1b[18;1H \x1b[m\x1b[18;79H\x1b[m\x1b[18;80H \x1b[4;79H\x1b[22;1H The selected entry will be started automatically in 5s. \x1b[4;79H\x1b[03;00H \x1b[04;00H\x1b[0m\x1b[2;30;47m CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[0m\x1b[2;37;40m \x00\x1b[05;00H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[06;00H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H Use the * and * keys to change the selection. \x1b[22;00H Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[22;1H The selected entry will be started automatically in 4s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 4s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 3s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 3s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 2s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 2s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 1s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 1s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 0s. \x1b[4;79H\x1b[?25h\x1b[H\x1b[J\x1b[1;1H\x1b[H\x1b[J\x1b[1;1H\x1b[04;00H \x1b[05;00H \x1b[06;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[01;01H[ 0.000000] Linux version 4.18.0-193.28.1.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Thu Oct 22 00:20:22 UTC 2020\r\n[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: \'x87 floating point registers\'\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: \'SSE registers\'\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: \'AVX registers\'\r\n[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256\r\n[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using \'standard\' format.\r\n[ 0.000000] BIOS-provided physical RAM map:\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bbff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078e74fff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x0000000078f24000-0x00000000798adfff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000798ae000-0x0000000079d4afff] ACPI NVS\r\n[ 0.000000] BIOS-e820: [mem 0x0000000079d4b000-0x000000008fffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable\r\n[ 0.000000] NX (Execute Disable) protection: active\r\n[ 0.000000] SMBIOS 3.0 present.\r\n[ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0 12/17/2015\r\n[ 0.000000] tsc: Fast TSC calibration using PIT\r\n[ 0.000000] tsc: Detected 3499.708 MHz processor\r\n[ 0.000000] last_pfn = 0x880000 max_arch_pfn = 0x400000000\r\n[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT \r\n[ 0.000000] last_pfn = 0x78e75 max_arch_pfn = 0x400000000\r\n[ 0.000000] found SMP MP-table at [mem 0x000fda20-0x000fda2f]\r\n[ 0.000000] kexec: Reserving the low 1M of memory for crashkernel\r\n[ 0.000000] Using GB pages for direct mapping\r\n[ 0.000000] RAMDISK: [mem 0x34e8b000-0x3673dfff]\r\n[ 0.000000] ACPI: Early table checksum verification disabled\r\n[ 0.000000] ACPI: RSDP 0x00000000000F0580 000024 (v02 SUPERM)\r\n[ 0.000000] ACPI: XSDT 0x00000000798AE098 0000B4 (v01 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FACP 0x00000000798E15B0 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: DSDT 0x00000000798AE1E8 0333C4 (v02 SUPERM SMCI--MB 01072009 INTL 20091013)\r\n[ 0.000000] ACPI: FACS 0x0000000079D49F80 000040\r\n[ 0.000000] ACPI: APIC 0x00000000798E16C0 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FPDT 0x00000000798E1788 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FIDT 0x00000000798E17D0 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: SPMI 0x00000000798E1870 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000)\r\n[ 0.000000] ACPI: MCFG 0x00000000798E18B0 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097)\r\n[ 0.000000] ACPI: UEFI 0x00000000798E18F0 000042 (v01 SUPERM SMCI--MB 01072009 00000000)\r\n[ 0.000000] ACPI: HPET 0x00000000798E1938 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: WDDT 0x00000000798E1970 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013)\r\n[ 0.000000] ACPI: SSDT 0x00000000798E19B0 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798F8B30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798FB180 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: PRAD 0x00000000798FB1E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: DMAR 0x00000000798FB2F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: HEST 0x00000000798FB3B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: BERT 0x00000000798FB630 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: ERST 0x00000000798FB660 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: EINJ 0x00000000798FB890 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] No NUMA configuration found\r\n[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000087fffffff]\r\n[ 0.000000] NODE_DATA(0) allocated [mem 0x87ffd6000-0x87fffffff]\r\n[ 0.000000] Using crashkernel=auto, the size chosen is a best effort estimation.\r\n[ 0.000000] Reserving 160MB of memory at 1760MB for crashkernel (System RAM: 32653MB)\r\n[ 0.000000] Zone ranges:\r\n[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]\r\n[ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]\r\n[ 0.000000] Normal [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Device empty\r\n[ 0.000000] Movable zone start for each node\r\n[ 0.000000] Early memory node ranges\r\n[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000008afff]\r\n[ 0.000000] node 0: [mem 0x0000000000100000-0x0000000078e74fff]\r\n[ 0.000000] node 0: [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Zeroed struct page in unavailable ranges: 29185 pages\r\n[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff]\r\n[ 0.000000] ACPI: PM-Timer IO Port: 0x408\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])\r\n[ 0.000000] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23\r\n[ 0.000000] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)\r\n[ 0.000000] Using ACPI (MADT) for SMP configuration information\r\n[ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000\r\n[ 0.000000] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0xb000020 (or later)\r\n[ 0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x0008b000-0x000dffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78e75000-0x78f23fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78f24000-0x798adfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x798ae000-0x79d4afff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x79d4b000-0x8fffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x90000000-0xfed1bfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed44fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed45000-0xfeffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]\r\n[ 0.000000] [mem 0x90000000-0xfed1bfff] available for PCI devices\r\n[ 0.000000] Booting paravirtualized kernel on bare hardware\r\n[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns\r\n[ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1\r\n[ 0.000000] percpu: Embedded 53 pages/cpu s180224 r8192 d28672 u262144\r\n[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 8228667\r\n[ 0.000000] Policy zone: Normal\r\n[ 0.000000] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200\r\n[ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://access.redhat.com/ecosystem.\r\n[ 0.000000] Memory: 1856264K/33437692K available (12292K kernel code, 2152K rwdata, 3852K rodata, 2408K init, 6468K bss, 816892K reserved, 0K cma-reserved)\r\n[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1\r\n[ 0.000000] Kernel/User page tables isolation: enabled\r\n[ 0.000000] ftrace: allocating 36345 entries in 142 pages\r\n[ 0.001000] rcu: Hierarchical RCU implementation.\r\n[ 0.001000] rcu: \tRCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.\r\n[ 0.001000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.\r\n[ 0.001000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8\r\n[ 0.001000] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16\r\n[ 0.001000] random: get_random_bytes called from start_kernel+0x36b/0x55b with crng_init=0\r\n[ 0.001000] Console: colour VGA+ 80x25\r\n[ 0.001000] printk: console [tty0] enabled\r\n[ 0.001000] printk: console [ttyS1] enabled\r\n[ 0.001000] ACPI: Core revision 20190703\r\n[ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns\r\n[ 0.001000] APIC: Switch to symmetric I/O mode setup\r\n[ 0.001003] DMAR: Host address width 46\r\n[ 0.002002] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1\r\n[ 0.003004] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de\r\n[ 0.004001] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff\r\n[ 0.005001] DMAR: ATSR flags: 0x0\r\n[ 0.006001] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0\r\n[ 0.007002] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.008001] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.009001] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000\r\n[ 0.010001] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.\r\n[ 0.010002] DMAR-IR: Use \'intremap=no_x2apic_optout\' to override the BIOS setting.\r\n[ 0.012257] DMAR-IR: Enabled IRQ remapping in xapic mode\r\n[ 0.013000] x2apic: IRQ remapping doesn\'t support X2APIC mode\r\n[ 0.014432] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1\r\n[ 0.019001] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x32723d995b0, max_idle_ns: 440795369662 ns\r\n[ 0.020001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.41 BogoMIPS (lpj=3499708)\r\n[ 0.021001] pid_max: default: 32768 minimum: 301\r\n[ 0.022017] Security Framework initialized\r\n[ 0.023000] Yama: becoming mindful.\r\n[ 0.024002] SELinux: Initializing.\r\n[ 0.029820] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)\r\n[ 0.032444] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)\r\n[ 0.034010] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.035070] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.036187] CPU0: Thermal monitoring enabled (TM1)\r\n[ 0.037035] process: using mwait in idle threads\r\n[ 0.038002] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8\r\n[ 0.039000] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4\r\n[ 0.040001] FEATURE SPEC_CTRL Not Present\r\n[ 0.041000] FEATURE IBPB_SUPPORT Not Present\r\n[ 0.042000] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization\r\n[ 0.043001] Spectre V2 : Mitigation: Full generic retpoline\r\n[ 0.044000] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\r\n[ 0.045000] Speculative Store Bypass: Vulnerable\r\n[ 0.046003] TAA: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.047000] MDS: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.048215] Freeing SMP alternatives memory: 32K\r\n[ 0.052000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1)\r\n[ 0.052060] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver.\r\n[ 0.053001] ... version: 3\r\n[ 0.054000] ... bit width: 48\r\n[ 0.055000] ... generic registers: 4\r\n[ 0.056000] ... value mask: 0000ffffffffffff\r\n[ 0.057000] ... max period: 00007fffffffffff\r\n[ 0.058000] ... fixed-purpose events: 3\r\n[ 0.059000] ... event mask: 000000070000000f\r\n[ 0.060026] rcu: Hierarchical SRCU implementation.\r\n[ 0.061759] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.\r\n[ 0.062014] smp: Bringing up secondary CPUs ...\r\n[ 0.064029] x86: Booting SMP configuration:\r\n[ 0.065001] .... node #0, CPUs: #1 #2 #3 #4\r\n[ 0.067667] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.\r\n[ 0.069001] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details.\r\n[ 0.070071] #5 #6 #7\r\n[ 0.072188] smp: Brought up 1 node, 8 CPUs\r\n[ 0.074001] smpboot: Max logical packages: 1\r\n[ 0.075001] smpboot: Total of 8 processors activated (55995.32 BogoMIPS)\r\n[ 0.279007] node 0 initialised, 7691134 pages in 202ms\r\n[ 0.285361] devtmpfs: initialized\r\n[ 0.286034] x86/mm: Memory block size: 128MB\r\n[ 0.289131] PM: Registering ACPI NVS region [mem 0x798ae000-0x79d4afff] (4837376 bytes)\r\n[ 0.290080] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns\r\n[ 0.301013] futex hash table entries: 2048 (order: 5, 131072 bytes)\r\n[ 0.307059] pinctrl core: initialized pinctrl subsystem\r\n[ 0.313111] NET: Registered protocol family 16\r\n[ 0.318067] audit: initializing netlink subsys (disabled)\r\n[ 0.324010] audit: type=2000 audit(1605565577.324:1): state=initialized audit_enabled=0 res=1\r\n[ 0.333002] cpuidle: using governor menu\r\n[ 0.337022] ACPI FADT declares the system doesn\'t support PCIe ASPM, so disable it\r\n[ 0.345001] ACPI: bus type PCI registered\r\n[ 0.350001] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5\r\n[ 0.356039] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)\r\n[ 0.366002] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820\r\n[ 0.373010] PCI: Using configuration type 1 for base access\r\n[ 0.381006] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages\r\n[ 0.388001] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages\r\n[ 0.395020] cryptd: max_cpu_qlen set to 1000\r\n[ 0.399067] ACPI: Added _OSI(Module Device)\r\n[ 0.404001] ACPI: Added _OSI(Processor Device)\r\n[ 0.409000] ACPI: Added _OSI(3.0 _SCP Extensions)\r\n[ 0.414000] ACPI: Added _OSI(Processor Aggregator Device)\r\n[ 0.419001] ACPI: Added _OSI(Linux-Dell-Video)\r\n[ 0.424000] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)\r\n[ 0.430000] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)\r\n[ 0.459727] ACPI: 4 ACPI AML tables successfully acquired and loaded\r\n[ 0.468813] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored\r\n[ 0.478086] ACPI: Dynamic OEM Table Load:\r\n[ 0.495355] ACPI: Interpreter enabled\r\n[ 0.500011] ACPI: (supports S0 S4 S5)\r\n[ 0.504000] ACPI: Using IOAPIC for interrupt routing\r\n[ 0.509019] HEST: Enabling Firmware First mode for corrected errors.\r\n[ 0.516071] HEST: Table parsing has been initialized.\r\n[ 0.521001] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug\r\n[ 0.531475] ACPI: Enabled 5 GPEs in block 00 to 3F\r\n[ 0.556832] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff])\r\n[ 0.563003] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.574133] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.583534] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.591001] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 0.600026] PCI host bridge to bus 0000:ff\r\n[ 0.605001] pci_bus 0000:ff: root bus resource [bus ff]\r\n[ 0.610009] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000\r\n[ 0.617038] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100\r\n[ 0.623028] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100\r\n[ 0.629027] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000\r\n[ 0.636028] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000\r\n[ 0.642026] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000\r\n[ 0.648027] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000\r\n[ 0.655027] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000\r\n[ 0.661028] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000\r\n[ 0.667026] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000\r\n[ 0.674027] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000\r\n[ 0.680026] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000\r\n[ 0.687026] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000\r\n[ 0.693027] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000\r\n[ 0.699026] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100\r\n[ 0.706027] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000\r\n[ 0.712027] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100\r\n[ 0.718026] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000\r\n[ 0.725026] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000\r\n[ 0.731018] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100\r\n[ 0.737031] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000\r\n[ 0.744055] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000\r\n[ 0.750037] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000\r\n[ 0.757036] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000\r\n[ 0.763037] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000\r\n[ 0.769036] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000\r\n[ 0.776042] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000\r\n[ 0.782030] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000\r\n[ 0.788031] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000\r\n[ 0.795037] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000\r\n[ 0.801036] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000\r\n[ 0.808036] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000\r\n[ 0.814036] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000\r\n[ 0.820029] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000\r\n[ 0.827028] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000\r\n[ 0.833028] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000\r\n[ 0.839029] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000\r\n[ 0.846037] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000\r\n[ 0.852036] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000\r\n[ 0.858038] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000\r\n[ 0.865039] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000\r\n[ 0.871033] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000\r\n[ 0.878030] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000\r\n[ 0.884029] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000\r\n[ 0.890032] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000\r\n[ 0.897028] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000\r\n[ 0.903028] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000\r\n[ 0.909031] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000\r\n[ 0.916032] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000\r\n[ 0.922028] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000\r\n[ 0.929027] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000\r\n[ 0.935027] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000\r\n[ 0.941019] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000\r\n[ 0.948029] pci 0000:ff:1f.0: [8086:6f88] type 00 class 0x088000\r\n[ 0.954027] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000\r\n[ 0.962213] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])\r\n[ 0.969002] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.979181] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.987241] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.995002] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 1.004195] PCI host bridge to bus 0000:00\r\n[ 1.009001] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]\r\n[ 1.016001] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window]\r\n[ 1.023002] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]\r\n[ 1.031001] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window]\r\n[ 1.039001] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window]\r\n[ 1.047001] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window]\r\n[ 1.055001] pci_bus 0000:00: root bus resource [bus 00-fe]\r\n[ 1.061006] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000\r\n[ 1.068102] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400\r\n[ 1.074045] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold\r\n[ 1.081072] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400\r\n[ 1.087044] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold\r\n[ 1.093069] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400\r\n[ 1.100044] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold\r\n[ 1.106052] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000\r\n[ 1.113009] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit]\r\n[ 1.120086] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000\r\n[ 1.126008] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit]\r\n[ 1.133086] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000\r\n[ 1.140008] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit]\r\n[ 1.147086] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000\r\n[ 1.153008] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit]\r\n[ 1.160087] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000\r\n[ 1.167008] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit]\r\n[ 1.174087] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000\r\n[ 1.180008] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit]\r\n[ 1.188085] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000\r\n[ 1.194008] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit]\r\n[ 1.201086] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000\r\n[ 1.208008] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit]\r\n[ 1.215085] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000\r\n[ 1.221081] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000\r\n[ 1.227092] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000\r\n[ 1.234081] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020\r\n[ 1.240006] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff]\r\n[ 1.247093] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000\r\n[ 1.253163] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601\r\n[ 1.260016] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117]\r\n[ 1.266006] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103]\r\n[ 1.272006] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7]\r\n[ 1.278006] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3]\r\n[ 1.284006] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f]\r\n[ 1.289006] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff]\r\n[ 1.296035] pci 0000:00:11.4: PME# supported from D3hot\r\n[ 1.302073] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330\r\n[ 1.308017] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit]\r\n[ 1.315050] pci 0000:00:14.0: PME# supported from D3hot D3cold\r\n[ 1.321065] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000\r\n[ 1.328017] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit]\r\n[ 1.335051] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold\r\n[ 1.341064] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000\r\n[ 1.348017] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit]\r\n[ 1.363051] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold\r\n[ 1.369074] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320\r\n[ 1.376018] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff]\r\n[ 1.382073] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold\r\n[ 1.389070] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400\r\n[ 1.395068] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold\r\n[ 1.402073] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400\r\n[ 1.408069] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold\r\n[ 1.414073] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400\r\n[ 1.421068] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold\r\n[ 1.427077] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320\r\n[ 1.434018] pci 0000:00:1d.0: reg 0x10: [mem 0xfb433000-0xfb4333ff]\r\n[ 1.440072] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold\r\n[ 1.447071] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100\r\n[ 1.453108] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601\r\n[ 1.460013] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077]\r\n[ 1.466005] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063]\r\n[ 1.472005] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057]\r\n[ 1.477005] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043]\r\n[ 1.483005] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f]\r\n[ 1.489005] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff]\r\n[ 1.496029] pci 0000:00:1f.2: PME# supported from D3hot\r\n[ 1.501064] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500\r\n[ 1.508015] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit]\r\n[ 1.515016] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f]\r\n[ 1.521169] acpiphp: Slot [1] registered\r\n[ 1.525013] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 1.531106] pci 0000:02:00.0: [8086:0953] type 00 class 0x010802\r\n[ 1.537012] pci 0000:02:00.0: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit]\r\n[ 1.544016] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref]\r\n[ 1.551003] pci 0000:02:00.0: enabling Extended Tags\r\n[ 1.557074] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 1.562002] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 1.569107] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000\r\n[ 1.576013] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref]\r\n[ 1.583004] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f]\r\n[ 1.589010] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref]\r\n[ 1.597004] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref]\r\n[ 1.604036] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.611016] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.618002] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.629008] pci 0000:03:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.637001] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.648117] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000\r\n[ 1.655013] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref]\r\n[ 1.662004] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f]\r\n[ 1.668010] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref]\r\n[ 1.676004] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref]\r\n[ 1.683035] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.690014] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.698001] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.708008] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.716001] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.727076] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 1.733001] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 1.739001] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 1.746002] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 1.754028] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 1.760067] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400\r\n[ 1.766060] pci 0000:05:00.0: enabling Extended Tags\r\n[ 1.772064] pci 0000:05:00.0: supports D1 D2\r\n[ 1.773001] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.774133] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 1.775002] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 1.776002] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.777048] pci_bus 0000:06: extended config space not accessible\r\n[ 1.778026] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000\r\n[ 1.779023] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff]\r\n[ 1.780012] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff]\r\n[ 1.781012] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f]\r\n[ 1.783014] pci 0000:06:00.0: supports D1 D2\r\n[ 1.784001] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.785092] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 1.786007] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 1.787004] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.788066] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000\r\n[ 1.789029] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff]\r\n[ 1.790018] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f]\r\n[ 1.791009] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff]\r\n[ 1.792114] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.793035] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.794001] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.795021] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.796001] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 1.798083] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000\r\n[ 1.799027] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff]\r\n[ 1.800018] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f]\r\n[ 1.801009] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff]\r\n[ 1.802109] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.803034] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.804001] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.805021] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.806001] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 1.807119] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 1.808002] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 1.809002] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 1.810004] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08\r\n[ 1.812020] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)\r\n[ 1.813031] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15)\r\n[ 1.814030] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)\r\n[ 1.815030] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)\r\n[ 1.816030] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.817030] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.818029] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.819029] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.820081] iommu: Default domain type: Passthrough \r\n[ 1.826018] pci 0000:06:00.0: vgaarb: setting as boot VGA device\r\n[ 1.827000] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none\r\n[ 1.841001] pci 0000:06:00.0: vgaarb: bridge control possible\r\n[ 1.847000] vgaarb: loaded\r\n[ 1.850057] SCSI subsystem initialized\r\n[ 1.854013] ACPI: bus type USB registered\r\n[ 1.859010] usbcore: registered new interface driver usbfs\r\n[ 1.864005] usbcore: registered new interface driver hub\r\n[ 1.870016] usbcore: registered new device driver usb\r\n[ 1.875018] pps_core: LinuxPPS API ver. 1 registered\r\n[ 1.881000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti \r\n[ 1.891002] PTP clock support registered\r\n[ 1.895014] EDAC MC: Ver: 3.0.0\r\n[ 1.898100] PCI: Using ACPI for IRQ routing\r\n[ 1.908000] NetLabel: Initializing\r\n[ 1.912001] NetLabel: domain hash size = 128\r\n[ 1.917000] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO\r\n[ 1.923009] NetLabel: unlabeled traffic allowed by default\r\n[ 1.929034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0\r\n[ 1.935001] hpet0: 8 comparators, 64-bit 14.318180 MHz counter\r\n[ 1.943019] clocksource: Switched to clocksource tsc-early\r\n[ 1.957109] VFS: Disk quotas dquot_6.6.0\r\n[ 1.961377] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)\r\n[ 1.968626] pnp: PnP ACPI init\r\n[ 1.972314] system 00:01: [io 0x0500-0x057f] has been reserved\r\n[ 1.978564] system 00:01: [io 0x0400-0x047f] has been reserved\r\n[ 1.984814] system 00:01: [io 0x0580-0x059f] has been reserved\r\n[ 1.991062] system 00:01: [io 0x0600-0x061f] has been reserved\r\n[ 1.997312] system 00:01: [io 0x0880-0x0883] has been reserved\r\n[ 2.003561] system 00:01: [io 0x0800-0x081f] has been reserved\r\n[ 2.009810] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved\r\n[ 2.016755] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved\r\n[ 2.023696] system 00:01: [mem 0xff000000-0xffffffff] has been reserved\r\n[ 2.030641] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved\r\n[ 2.037582] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved\r\n[ 2.044524] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved\r\n[ 2.051468] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved\r\n[ 2.058545] system 00:02: [io 0x0a00-0x0a0f] has been reserved\r\n[ 2.064799] system 00:02: [io 0x0a10-0x0a1f] has been reserved\r\n[ 2.071047] system 00:02: [io 0x0a20-0x0a2f] has been reserved\r\n[ 2.077296] system 00:02: [io 0x0a30-0x0a3f] has been reserved\r\n[ 2.083547] system 00:02: [io 0x0a40-0x0a4f] has been reserved\r\n[ 2.090458] pnp: PnP ACPI: found 5 devices\r\n[ 2.100188] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns\r\n[ 2.109690] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000\r\n[ 2.129497] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.137929] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 2.143224] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 2.148515] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 2.155642] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.163810] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.172321] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.180572] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.189172] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.197336] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.205848] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.214100] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.222698] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.230862] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.239375] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.247623] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.256226] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.264397] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.272911] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.281167] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.289767] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 2.295059] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 2.301485] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 2.308601] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.316941] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 2.322239] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 2.327537] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 2.333965] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.341096] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 2.346651] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 2.353075] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.360197] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref]\r\n[ 2.368542] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref]\r\n[ 2.376975] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref]\r\n[ 2.385322] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref]\r\n[ 2.393757] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 2.399053] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 2.405481] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 2.412607] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.420955] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc\r\n[ 2.430601] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]\r\n[ 2.437114] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window]\r\n[ 2.443624] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]\r\n[ 2.450828] pci_bus 0000:00: resource 7 [mem 0xfedb0000-0xfedb000f window]\r\n[ 2.458031] pci_bus 0000:00: resource 8 [mem 0xfedc0000-0xfedc000f window]\r\n[ 2.465235] pci_bus 0000:00: resource 9 [mem 0x90000000-0xfbffbfff window]\r\n[ 2.472437] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff]\r\n[ 2.479030] pci_bus 0000:03: resource 0 [io 0xe000-0xefff]\r\n[ 2.484933] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff]\r\n[ 2.491527] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.499348] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff]\r\n[ 2.505277] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.511873] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff]\r\n[ 2.517772] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.524372] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff]\r\n[ 2.530271] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff]\r\n[ 2.536870] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.544781] NET: Registered protocol family 2\r\n[ 2.549567] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes)\r\n[ 2.558251] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)\r\n[ 2.566479] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)\r\n[ 2.573615] TCP: Hash tables configured (established 262144 bind 65536)\r\n[ 2.580606] UDP hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.587180] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.594226] NET: Registered protocol family 1\r\n[ 2.598921] NET: Registered protocol family 44\r\n[ 2.621075] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x6c5 took 16773 usecs\r\n[ 2.646072] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x6c5 took 16596 usecs\r\n[ 2.654073] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes\r\n[ 2.660855] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]\r\n[ 2.669836] Unpacking initramfs...\r\n[ 2.930133] Freeing initrd memory: 25292K\r\n[ 2.940027] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\r\n[ 2.946806] software IO TLB: mapped [mem 0x6a000000-0x6e000000] (64MB)\r\n[ 2.962425] Initialise system trusted keyrings\r\n[ 2.967211] Key type blacklist registered\r\n[ 2.971575] workingset: timestamp_bits=36 max_order=23 bucket_order=0\r\n[ 2.979292] zbud: loaded\r\n[ 2.982424] pstore: using deflate compression\r\n[ 2.987290] Platform Keyring initialized\r\n[ 3.049541] NET: Registered protocol family 38\r\n[ 3.054313] Key type asymmetric registered\r\n[ 3.058740] Asymmetric key parser \'x509\' registered\r\n[ 3.063960] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)\r\n[ 3.071978] io scheduler mq-deadline registered\r\n[ 3.076839] io scheduler kyber registered\r\n[ 3.081200] io scheduler bfq registered\r\n[ 3.085417] atomic64_test: passed for x86-64 platform with CX8 and with SSE\r\n[ 3.092956] pcieport 0000:00:01.0: Signaling with IRQ 25\r\n[ 3.098748] pcieport 0000:00:02.0: Signaling with IRQ 27\r\n[ 3.104524] pcieport 0000:00:03.0: Signaling with IRQ 29\r\n[ 3.110286] pcieport 0000:00:1c.0: Signaling with IRQ 30\r\n[ 3.116060] pcieport 0000:00:1c.2: Signaling with IRQ 31\r\n[ 3.121825] pcieport 0000:00:1c.4: Signaling with IRQ 32\r\n[ 3.127548] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4\r\n[ 3.135112] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0\r\n[ 3.144066] ACPI: Power Button [PWRB]\r\n[ 3.148078] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1\r\n[ 3.156079] ACPI: Power Button [PWRF]\r\n[ 3.161113] ERST: Error Record Serialization Table (ERST) support is initialized.\r\n[ 3.169227] pstore: Registered erst as persistent store backend\r\n[ 3.175630] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.\r\n[ 3.183719] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled\r\n[ 3.210903] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A\r\n[ 3.239433] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A\r\n[ 3.247647] Non-volatile memory driver v1.3\r\n[ 3.252542] rdac: device handler registered\r\n[ 3.257102] hp_sw: device handler registered\r\n[ 3.261703] emc: device handler registered\r\n[ 3.266152] alua: device handler registered\r\n[ 3.270685] libphy: Fixed MDIO Bus: probed\r\n[ 3.275164] ehci_hcd: USB 2.0 \'Enhanced\' Host Controller (EHCI) Driver\r\n[ 3.282025] ehci-pci: EHCI PCI platform driver\r\n[ 3.286874] ehci-pci 0000:00:1a.0: EHCI Host Controller\r\n[ 3.292465] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1\r\n[ 3.300468] ehci-pci 0000:00:1a.0: debug port 2\r\n[ 3.309222] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported\r\n[ 3.316347] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000\r\n[ 3.329066] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00\r\n[ 3.335171] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.344033] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.351849] usb usb1: Product: EHCI Host Controller\r\n[ 3.357054] usb usb1: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.364962] usb usb1: SerialNumber: 0000:00:1a.0\r\n[ 3.369979] hub 1-0:1.0: USB hub found\r\n[ 3.374063] hub 1-0:1.0: 2 ports detected\r\n[ 3.378548] ehci-pci 0000:00:1d.0: EHCI Host Controller\r\n[ 3.384137] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2\r\n[ 3.392137] ehci-pci 0000:00:1d.0: debug port 2\r\n[ 3.400908] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported\r\n[ 3.408026] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000\r\n[ 3.421062] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00\r\n[ 3.427169] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.436030] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.443846] usb usb2: Product: EHCI Host Controller\r\n[ 3.456748] usb usb2: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.464658] usb usb2: SerialNumber: 0000:00:1d.0\r\n[ 3.469665] hub 2-0:1.0: USB hub found\r\n[ 3.473747] hub 2-0:1.0: 2 ports detected\r\n[ 3.478172] ohci_hcd: USB 1.1 \'Open\' Host Controller (OHCI) Driver\r\n[ 3.484682] ohci-pci: OHCI PCI platform driver\r\n[ 3.489463] uhci_hcd: USB Universal Host Controller Interface driver\r\n[ 3.496236] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.501831] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3\r\n[ 3.510875] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810\r\n[ 3.520607] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported\r\n[ 3.527845] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.536706] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.544527] usb usb3: Product: xHCI Host Controller\r\n[ 3.549736] usb usb3: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.557644] usb usb3: SerialNumber: 0000:00:14.0\r\n[ 3.562698] hub 3-0:1.0: USB hub found\r\n[ 3.566786] hub 3-0:1.0: 15 ports detected\r\n[ 3.572632] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.578217] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4\r\n[ 3.586214] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed\r\n[ 3.592913] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 4.18\r\n[ 3.601771] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.609587] usb usb4: Product: xHCI Host Controller\r\n[ 3.614795] usb usb4: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.622701] usb usb4: SerialNumber: 0000:00:14.0\r\n[ 3.627717] hub 4-0:1.0: USB hub found\r\n[ 3.631814] hub 4-0:1.0: 6 ports detected\r\n[ 3.636877] usbcore: registered new interface driver usbserial_generic\r\n[ 3.643746] usbserial: USB Serial support registered for generic\r\n[ 3.650109] i8042: PNP: No PS/2 controller found.\r\n[ 3.655185] mousedev: PS/2 mouse device common for all mice\r\n[ 3.661147] rtc_cmos 00:00: RTC can wake from S4\r\n[ 3.666221] rtc_cmos 00:00: registered as rtc0\r\n[ 3.671004] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram, hpet irqs\r\n[ 3.679285] intel_pstate: Intel P-state driver initializing\r\n[ 3.685972] hidraw: raw HID events driver (C) Jiri Kosina\r\n[ 3.691729] usbcore: registered new interface driver usbhid\r\n[ 3.697633] usbhid: USB HID core driver\r\n[ 3.701869] drop_monitor: Initializing network drop monitor service\r\n[ 3.708509] Initializing XFRM netlink socket\r\n[ 3.713182] NET: Registered protocol family 10\r\n[ 3.718148] Segment Routing with IPv6\r\n[ 3.722150] NET: Registered protocol family 17\r\n[ 3.726970] mpls_gso: MPLS GSO support\r\n[ 3.731424] core: Using 22 MCE banks\r\n[ 3.735351] microcode: sig=0x406f1, pf=0x1, revision=0xb000010\r\n[ 3.741524] usb 1-1: new high-speed USB device number 2 using ehci-pci\r\n[ 3.741627] microcode: Microcode Update Driver: v2.2.\r\n[ 3.748503] resctrl: L3 allocation detected\r\n[ 3.758399] resctrl: L3DATA allocation detected\r\n[ 3.763261] resctrl: L3CODE allocation detected\r\n[ 3.768121] resctrl: L3 monitoring detected\r\n[ 3.772646] AVX2 version of gcm_enc/dec engaged.\r\n[ 3.777598] AES CTR mode by8 optimization enabled\r\n[ 3.807496] sched_clock: Marking stable (3807490802, 0)->(5630292005, -1822801203)\r\n[ 3.815821] registered taskstats version 1\r\n[ 3.820274] Loading compiled-in X.509 certificates\r\n[ 3.850036] usb 2-1: new high-speed USB device number 2 using ehci-pci\r\n[ 3.850505] Loaded X.509 cert \'CentOS kernel signing key: dbeb60232b9048c59969054d0225be669c9ec38b\'\r\n[ 3.866608] Loaded X.509 cert \'CentOS Linux Driver update signing key: 29bd4c0d06d2e9911044b5dc973309139b51d6d5\'\r\n[ 3.876421] usb 1-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05\r\n[ 3.877441] Loaded X.509 cert \'CentOS Linux kpatch signing key: b49f086205909dc4da2cfa99376fb191d2f09e78\'\r\n[ 3.886227] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 3.896729] hub 1-1:1.0: USB hub found\r\n[ 3.903935] zswap: loaded using pool lzo/zbud\r\n[ 3.908001] usb 3-12: new high-speed USB device number 2 using xhci_hcd\r\n[ 3.920045] hub 1-1:1.0: 6 ports detected\r\n[ 3.924617] Key type big_key registered\r\n[ 3.931012] Key type encrypted registered\r\n[ 3.935386] ima: No TPM chip found, activating TPM-bypass!\r\n[ 3.941220] ima: Allocated hash algorithm: sha1\r\n[ 3.946104] evm: Initialising EVM extended attributes:\r\n[ 3.951593] evm: security.selinux\r\n[ 3.955251] evm: security.ima\r\n[ 3.958563] evm: security.capability\r\n[ 3.962481] evm: HMAC attrs: 0x1\r\n[ 3.966439] rtc_cmos 00:00: setting system clock to 2020-11-16 22:26:23 UTC (1605565583)\r\n[ 3.975927] Freeing unused decrypted memory: 2040K\r\n[ 3.981390] Freeing unused kernel memory: 2408K\r\n[ 3.987426] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05\r\n[ 3.996219] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 3.999036] tsc: Refined TSC clocksource calibration: 3499.996 MHz\r\n[ 4.010246] Write protecting the kernel read-only data: 18432k\r\n[ 4.010247] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734d20948, max_idle_ns: 440795306379 ns\r\n[ 4.027012] clocksource: Switched to clocksource tsc\r\n[ 4.027035] hub 2-1:1.0: USB hub found\r\n[ 4.036557] hub 2-1:1.0: 8 ports detected\r\n[ 4.041157] Freeing unused kernel memory: 2020K\r\n[ 4.046087] Freeing unused kernel memory: 244K\r\n[ 4.048234] usb 3-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00\r\n[ 4.059726] usb 3-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.060301] hub 3-12:1.0: USB hub found\r\n[ 4.071838] hub 3-12:1.0: 4 ports detected\r\n[ 4.076395] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 4.110085] systemd[1]: Detected architecture x86-64.\r\n[ 4.115473] systemd[1]: Running in initial RAM disk.\r\n\r\nWelcome to \x1b[0;34mCentOS Linux 8 (Core) dracut-049-27.git20190906.el8 (Initramfs)\x1b[0m!\r\n\r\n[ 4.132114] systemd[1]: Set hostname to .\r\n[ 4.167860] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.174871] systemd[1]: Listening on udev Control Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\n[ 4.187165] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.194118] systemd[1]: Reached target Swap.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\n[ 4.204089] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.211086] systemd[1]: Listening on udev Kernel Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Kernel Socket.\r\n[ 4.223151] systemd[1]: Reached target Slices.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\n[ 4.233309] systemd[1]: Listening on Journal Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on Journal Socket.\r\n[ 4.245785] systemd[1]: Starting Setup Virtual Console...\r\n Starting Setup Virtual Console...\r\n Starting Create list of required st\xe2\x80\xa6ce nodes for the current kernel...\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\n Starting Apply Kernel Variables...\r\n[\x1b[0;32m OK \x1b[0m] Listening on Startinvice...\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\n Starting Create Volatile Files and Directories...\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directorie[ 4.372006] usb 3-12.1: new low-speed USB device number 3 using xhci_hcd\r\ns.\r\n Starting Creat..\r\n Stat cmdline hook...\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\n[\x1b[0;32m OK \x1b[0m] Started dracut cmdline hook.\r\n Starting dracut pre-udev hook...\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-udev hook.\r\n Starting udev K[ 4.486275] usb 3-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00\r\n[ 4.495316] usb 3-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.521100] i HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.0/0003:0557:2419.0001/input/input2\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\n[ 4.598104] heric 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0\r\n[ 4.623731] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.1/0003:0557:2419.0002/input/input3\r\n[ 4.648046] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\n Starting udev Coldplug all Devices...\r\n Mounting Kernel Configuration File System...\r\n[\x1b[0;32m OK \x1b[0m] Mounted Kernel Configuration File System.\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\n Starting Show Plymouth Boot Screen...\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\n Starting dracut initque[ 4.910910] dca service started, version 1.12.1\r\nue hook...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Show Plymouth Boot Screen.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Password Requests to Plymouth Directory[ 4.930925] nvme nvme0: pci function 0000:02:00.0\r\n Watch.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Basic System.\r\r\n[ 4.949384] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode\r\n[ 4.958107] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 4.966655] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k\r\n[ 4.973958] igb: Copyright (c) 2007-2014 Intel Corporation.\r\n[ 4.992114] scsi host0: ahci\r\n[ 5.003356] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k-rh8.2.0\r\n[ 5.005716] scsi host1: ahci\r\n[ 5.012315] ixgbe: Copyright (c) 1999-2016 Intel Corporation.\r\n[ 5.044438] igb 0000:07:00.0: added PHC on eth0\r\n[ 5.049306] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.056525] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:e4\r\n[ 5.056767] nvme nvme0: 8/0/0 default/read/poll queues\r\n[ 5.064077] igb 0000:07:00.0: eth0: PBA No: 050B00-000\r\n[ 5.074955] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)\r\n[ 5.083151] scsi host2: ahci\r\n[ 5.107017] scsi host3: ahci\r\n[ 5.110318] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 35\r\n[ 5.118324] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 35\r\n[ 5.126348] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 35\r\n[ 5.134353] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 35\r\n[ 5.173791] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode\r\n[ 5.190277] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 5.198543] [drm] Using P2A bridge for configuration\r\n[ 5.203853] [drm] AST 2400 detected\r\n[ 5.207705] [drm] Analog VGA only\r\n[ 5.211378] [drm] dram MCLK=408 Mhz type=1 bus_width=16 size=01000000\r\n[ 5.250227] [TTM] Zone kernel: Available graphics memory: 16326418 KiB\r\n[ 5.257191] [TTM] Zone dma32: Available graphics memory: 2097152 KiB\r\n[ 5.264058] [TTM] Initializing pool allocator\r\n[ 5.273431] igb 0000:07:00.1: added PHC on eth1\r\n[ 5.278305] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.285527] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:e5\r\n[ 5.293087] igb 0000:07:00.1: eth1: PBA No: 050B00-000\r\n[ 5.298578] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)\r\n[ 5.306772] scsi host4: ahci\r\n[ 5.322014] scsi host5: ahci\r\n[ 5.325349] [TTM] Initializing DMA pool allocator\r\n[ 5.336394] scsi host6: ahci\r\n[ 5.347973] scsi host7: ahci\r\n[ 5.353987] fbcon: astdrmfb (fb0) is primary device\r\n[ 5.377436] scsi host8: ahci\r\n[ 5.390019] scsi host9: ahci\r\n[ 5.390092] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 63\r\n[ 5.390093] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 63\r\n[ 5.390095] ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 63\r\n[ 5.390096] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 63\r\n[ 5.390098] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 63\r\n[ 5.390099] ata10: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 63\r\n[ 5.443015] Console: switching to colour frame buffer device 128x48\r\n[ 5.454025] ata4: SATA link down (SStatus 0 SControl 300)\r\n[ 5.454221] ata2: SATA link down (SStatus 0 SControl 300)\r\n[ 5.454388] ata3: SATA link down (SStatus 0 SControl 300)\r\n[ 5.454436] ata1: SATA link down (SStatus 0 SControl 300)\r\n[ 5.591494] igb 0000:07:00.0 eno1: renamed from eth0\r\n[ 5.599594] igb 0000:07:00.1 eno2: renamed from eth1\r\n[ 5.605303] ast 0000:06:00.0: fb0: astdrmfb frame buffer device\r\n[ 5.688010] [drm] Initialized ast 0.1.0 20120228 for 0000:06:00.0 on minor 0\r\n[ 5.712171] ata5: SATA lintatus 133 SControl 300)\r\n[ 5.728517] ata6: SATA link down (SStatus 0 SControl 300)\r\n[ 5.733947] ata7: SATA link down (SStatus 0 SControl 300)\r\n[ 5.733966] ata9: SATA link down (SStatus 0 SControl 300)\r\n[ 5.744809] ata8: SATA link down (SStatus 0 SControl 300)\r\n[ 5.744834] ata10: SATA link down (SStatus 0 SControl 300)\r\n[ 5.768554] ata5.00: NCQ Send/Recv Log not supported\r\n[ 5.768555] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133\r\n[ 5.768557] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 32), AA\r\n[ 5.770243] ata5.00: NCQ Send/Recv Log not supported\r\n[ 5.770246] ata5.00: configured for UDMA/133\r\n[ 5.780456] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5\r\n[ 5.787669] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)\r\n[ 5.787676] sd 4:0:0:0: [sda] Write Protect is off\r\n[ 5.787688] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn\'t support DPO or FUA\r\n[ 5.803827] sda: sda1\r\n[ 5.859973] sd 4:0:0:0: [sda] Attached SCSI disk\r\n[\x1b[0;32m OK \x1b[0m] Found device ST1000NM0033-9ZM173 1.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root Device.\r\r\n[ 6.061921] random: crng init done\r\n[ 6.065655] random: 7 urandom warning(s) missed due to ratelimiting\r\n[ 6.156170] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.166798] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.175064] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF\r\n[ 6.181586] ixgbe 0000:03:00.0: 0c:c4:7a:8f:d5:14\r\n[ 6.204375] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection\r\n[ 6.204436] libphy: ixgbe-mdio: probed\r\n[ 6.371137] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.381783] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.390062] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF\r\n[ 6.397487] ixgbe 0000:03:00.1: 0c:c4:7a:8f:d5:15\r\n[ 6.420283] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection\r\n[ 6.420301] libphy: ixgbe-mdio: probed\r\n[ 6.427908] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1\r\n[ 6.443388] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0\r\n[\x1b[0;32m OK \x1b[0m] Started dracut initqueue hook.\r\r\n Starting File System Check on /dev/\xe2\x80\xa68e7-5b1a-4776-884c-191bd8ad39f5...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started File System Check on /dev/d\xe2\x80\xa6f28e7-5b1a-4776-884c-191bd8ad39f5.\r\r\n Mounting /sysroot...\r\r\n[ 6.601460] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)\r\n[\x1b[0;32m OK \x1b[0m] Mounted /sysroot.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root File System.\r\r\n Starting Reload Configuration from the Real Root...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Reload Configuration from the Real Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Default Target.\r\r\n Starting dracut pre-pivot and cleanup hook...\r\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-pivot and cleanup hook.\r\r\n Starting Cleaning Up and Shutting Down Daemons...\r\r\n Starting Setup Virtual Console...\r\r\n Starting Plymouth switch root service...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-pivot and cleanup hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut initqueue hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Default Target.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root Device.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Basic System.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Swap.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Volatile Files and Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Apply Kernel Variables.\r\r\n Stopping udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Slices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Sockets.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Local File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleaning Up and Shutting Down Daemons.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Kernel Device Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Static Device Nodes in /dev.\r\r\n[ 7.001034] systemd-journald[326]: Received SIGTERM from PID 1 (systemd).\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-udev hook.[ 7.072527] printk: systemd: 21 output lines suppressed due to ratelimiting\r\n\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut cmdline hook.\r\r\n Stopping Hardware RNG Entropy Gatherer Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Control Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Kernel Socket.\r\r\n Starting Cleanup udevd DB...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Hardware RNG Entropy Gatherer Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Plymouth switch root service.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleanup udevd DB.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Switch Root.\r\r\n Starting Switch Root...\r\r\n[ 7.910028] SELinux: policy capability network_peer_controls=1\r\n[ 7.916284] SELinux: policy capability open_perms=1\r\n[ 7.921569] SELinux: policy capability extended_socket_class=1\r\n[ 7.927808] SELinux: policy capability always_check_network=0\r\n[ 7.933954] SELinux: policy capability cgroup_seclabel=1\r\n[ 7.939661] SELinux: policy capability nnp_nosuid_transition=1\r\n[ 7.961736] audit: type=1403 audit(1605565587.493:2): auid=4294967295 ses=4294967295 lsm=selinux res=1\r\n[ 7.963334] systemd[1]: Successfully loaded SELinux policy in 443.473ms.\r\n[ 8.179439] systemd[1]: Relabelled /dev, /run and /sys/fs/cgroup in 18.874ms.\r\n[ 8.188988] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 8.223152] systemd[1]: Detected architecture x86-64.\r\n\r\r\nWelcome to \x1b[0;31mCentOS Linux 8 (Core)\x1b[0m!\r\r\n\r\r\n[ 8.236938] systemd[1]: Set hostname to .\r\n[ 9.245659] systemd[1]: Stopped Switch Root.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Switch Root.\r\r\n[ 9.250758] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart.\r\n[ 9.250815] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.\r\n[ 9.250871] systemd[1]: Stopped Journal Service.\r\n[ 9.293521] systemd[1]: Starting Journal Service...\r\n[ 9.314883] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once.\r\n[ 9.349417] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Journal Service.\r\r\n Starting Journal Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mon@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mgr@.service instances at once.\r\r\n Mounting POSIX Message Queue File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Set up automount Arbitrary Executab\xe2\x80\xa6rmats File System Automount Point.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-serial\\x2dgetty.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-sshd\\x2dkeygen.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Device-mapper event daemon FIFOs.\r\r\n Starting Monitoring of LVM2 mirrors\xe2\x80\xa6ng dmeventd or progress polling...\r\r\n Mounting Kernel Debug File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on LVM2 poll daemon socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Process Core Dump Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on RPCbind Server Activation Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target RPC Port Mapper.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-getty.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6mirror@.service instances at once.\r\r\n [ 9.526388] EXT4-fs (sda1): re-mounted. Opts: (null)\r\n Starting Remount Root and Kernel File Systems...\r\r\n Mounting Huge Pages File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice User and Session Slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mds@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\r\n Starting Create list of required st\xe2\x80\xa6ce nodes for the current kernel...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Password Requests to Wall Directory Watch.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local Encrypted Volumes.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Kernel Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-osd@.service instances at once.\r[ 9.616181] device-mapper: uevent: version 1.0.3\r\n\r\n Start[ 9.616278] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com\r\ning Apply Kernel Variables...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6adosgw@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\r\n Starting udev Coldplug all Devices...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Switch Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd File Systems.\r\r\n Starting Setup Virtual Console...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on initctl Compatibility Named Pipe.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted POSIX Message Queue File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted Kernel Debug File[ 9.726069] systemd-journald[1436]: Received request to flush runtime journal from PID 1\r\n System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Monitoring of LVM2 mirrors,\xe2\x80\xa6sing dmeventd or progress polling.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Remount Root and Kernel File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted Huge Pages File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\r\n Starting Load/Save Random Seed...\r\r\n Starting Create Static Device Nodes in /dev...\r\r\n Starting Flush Journal to Persistent Storage...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load/Save Random Seed.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Flush Journal to Persistent Storage.\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\r\n Starting Import network configuration from initramfs...\r\r\n Starting Restore /run/initramfs on shutdown...\r\r\n Starting Tell Plymouth To Write Out Runtime Data...\r\r\n Starting udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Restore /run/initramfs on shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Tell Plymouth To Write Out Runtime Data.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Import network configuration from initramfs.\r\r\n Starting Create Volatile Files and Directories...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directories.\r\r\n Starting RPC Bind...\r\r\n Mounting RPC Pipe File System...\r\r\n Starting Security Auditing Service...\r\r\n[ 10.119620] RPC: Registered named UNIX socket transport module.\r\n[ 10.125946] RPC: Registered udp transport module.\r\n[ 10.131322] RPC: Registered tcp transport module.\r\n[ 10.136464] RPC: Registered tcp NFSv4.1 backchannel transport module.\r\n[\x1b[0;32m OK \x1b[0m] Mounted RPC Pipe File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target rpc_pipefs.target.\r\r\n[\x1b[0;32m OK \x1b[0m] Started RPC Bind.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Security Auditing Service.\r\r\n Starting Update UTMP about System Boot/Shutdown...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Update UTMP about System Boot/Shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Generate summary of yesterday\'s process accounting.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on SSSD Kerberos Cache Manager responder socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Daily Cleanup of Temporary Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on D-Bus System Message Bus Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Run system activity accounting tool every 10 minutes.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Basic System.\r\r\n Starting System Security Services Daemon...\r\r\n Starting Resets System Activity Logs...\r\r\n[ 10.483293] ioatdma: Intel(R) QuickData Technology Driver 5.00\r\n[ 10.497694] igb 0000:07:00.0: DCA enabled\r\n[ 10.502137] igb 0000:07:00.1: DCA enabled\r\n[\x1b[0;32m OK \x1b[0m] Started irqbalance daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Machine Check Exception Logging Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started libstoragemgmt plug-in server daemon.\r\r\n Starting GSSAPI Proxy Daemon...\r\r\n Starting VDO volume services...\r\r\n Starting Self Monitoring and Reporting Technology (SMART) Daemon...\r\r\n[ 10.567363] mei_me 0000:00:16.0: Device doesn\'t have valid ME Interface\r\n Starting Kernel Samepage Merging...\r\r\n[ 10.615202] power_meter ACPI000D:00: Found ACPI power meter.\r\n[ 10.621302] power_meter ACPI000D:00: Ignoring unsafe software power cap!\r\n Starting NTP client/server...\r\r\n[ 10.629793] IPMI message handler: version 39.2\r\n[ 10.637031] ipmi device interface\r\n[ 10.651354] ipmi_si: IPMI System Interface driver\r\n[ 10.665095] ipmi_si dmi-ipmi-si.0: probing via SMBIOS\r\n[ 10.665098] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0\r\n[ 10.665099] ipmi_si: Adding SMBIOS-specified kcs state machine\r\n[ 10.665140] ipmi_si IPI0001:00: probing via ACPI\r\n[ 10.665168] ipmi_si IPI0001:00: [io 0x0ca2] regsize 1 spacing 1 irq 0\r\n[ 10.665174] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI\r\n[ 10.665175] ipmi_si: Adding ACPI-specified kcs state machine\r\n[ 10.665246] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0\r\n[ 10.709011] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed.\r\n[ 10.760472] input: PC Speaker as /devices/platform/pcspkr/input/input4\r\n[ 10.761188] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt\r\n[ 10.765532] ipmi_si IPI0001:00: Found new BMC (man_id: 0x002a7c, prod_id: 0x0833, dev_id: 0x20)\r\n[ 10.766973] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer\r\n[ 10.766974] RAPL PMU: hw unit of domain package 2^-14 Joules\r\n[ 10.766974] RAPL PMU: hw unit of domain dram 2^-16 Joules\r\n[ 10.817607] ipmi_si IPI0001:00: IPMI kcs interface initialized\r\n[ 10.823668] ipmi_ssif: IPMI SSIF Interface driver\r\n Starting Load CPU microcode update...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started D-Bus System Message Bus.\r\r\n Starting Network Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target sshd-keygen.target.\r\r\n[\x1b[0;32m OK \x1b[0m] Started daily update of the root trust anchor for DNSSEC.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Ceph crash dump collector.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6[ 10.915572] sd 4:0:0:0: Attached scsi generic sg0 type 0\r\n ceph*@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Resets System Activity Logs.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-lvm2\\x2dpvscan.slice.\r\r\n Starting LVM event activation on device 259:0...\r\r\n Starting Kernel Samepage Merging (KSM) Tuning Daemon...\r\r\n[ 11.392584] iTCO_vendor_support: vendor-support=0\r\n[ 11.436930] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11\r\n[ 11.443051] iTCO_wdt: Found a Wellsburg TCO device (Version=2, TCOBASE=0x0460)\r\n[ 11.450963] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)\r\n[ 11.490021] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:ff:12.0 (INTERRUPT)\r\n[ 11.501474] EDAC sbridge: Ver: 1.1.2 \r\n[ 11.651623] intel_rapl_common: Found RAPL domain package\r\n[ 11.657452] intel_rapl_common: Found RAPL domain dram\r\n[ 11.663006] intel_rapl_common: DRAM domain energy unit 15300pj\r\n[\x1b[0;32m OK \x1b[0m] Started Self Monitoring and Reporting Technology (SMART) Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started GSSAPI Proxy Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load CPU microcode update.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging (KSM) Tuning Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started LVM event activation on device 259:0.\r\r\n[\x1b[0;32m OK \x1b[0m] Found device /dev/vg_nvme/lv_5.\r\r\n[\x1b[0;32m OK \x1b[0m] Started NTP client/server.\r\r\n Mounting /var/lib/ceph...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target NFS client services.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Network.\r\r\n[ 12.620913] SGI XFS with ACLs, security attributes, no debug enabled\r\n[ 12.631138] XFS (dm-4): Mounting V4 Filesystem\r\n[ 12.637701] XFS (dm-4): Ending clean mount\r\n Starting OpenSSH server daemon...\r\r\n Starting /etc/rc.d/rc.local Compatibility...\r\r\n+ \'[\' \'!\' -f /.cephlab_net_configured \']\'\r\n+ set +e\r\n+ attempts=0\r\n+ myips=\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 0 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n[\x1b[0;32m OK \x1b[0m] Started Nagios Remote Plugin Executor.\r\r\n Starting Network Manager Wait Online...\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted /var/lib/ceph.\r\r\n Starting Hostname Service...\r\r\n+ myips=\r\n+ attempts=1\r\n+ sleep 1\r\n[\x1b[0;32m OK \x1b[0m] Started OpenSSH server daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hostname Service.\r\r\n Starting Network Manager Script Dispatcher Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager Script Dispatcher Service.\r\r\n[ 13.198478] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.272199] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.281046] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[ 13.338458] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[\x1b[0;32m OK \x1b[0m] Started VDO volume services.\r\r\n[ 13.348114] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 13.387889] ixgbe 0000:03:00.0: registered PHC device on enp3s0f0\r\n[\x1b[0;32m OK \x1b[0m] Started System Security Services Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target User and Group Name Lookups.\r\r\n Starting Permit User Sessions...\r\r\n Starting Login Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Permit User Sessions.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Command Scheduler.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Job spooling tools.\r\r\n[ 13.496971] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 13.506941] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n[ 13.550697] ixgbe 0000:03:00.1: registered PHC device on enp3s0f1\r\n[\x1b[0;32m OK \x1b[0m] Started Login Service.\r\r\n[ 13.663961] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 1 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=2\r\n+ sleep 1\r\n[ 13.771562] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4\r\n[ 13.983164] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX\r\n[ 13.991341] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0f1: link becomes ready\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 2 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=3\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 3 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=4\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 4 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=5\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 5 -ge 10 \']\'\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ ip -4 addr\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=172.21.15.122\r\n+ attempts=6\r\n+ sleep 1\r\n+ \'[\' 172.21.15.122 \'!=\' \'\' \']\'\r\n+ set -e\r\n+ \'[\' -n 172.21.15.122 \']\'\r\n+ for ip in $myips\r\n+ timeout 1s ping -I 172.21.15.122 -nq -c1 172.21.0.1\r\n++ dig +short -x 172.21.15.122 @172.21.0.1\r\n++ sed \'s/\\.com.*/\\.com/g\'\r\n[\x1b[0m\x1b[0;31m* \x1b[0m] (1 of 2) A start job is running for\xe2\x80\xa6ocal Compatibility (9s / no limit)+ newhostname=smithi122.front.sepia.ceph.com\r\n+ \'[\' -n smithi122.front.sepia.ceph.com \']\'\r\n+ hostname smithi122.front.sepia.ceph.com\r\n++ hostname -d\r\n+ newdomain=front.sepia.ceph.com\r\n++ hostname -s\r\n+ shorthostname=smithi122\r\n+ echo smithi122\r\n+ grep -q front.sepia.ceph.com /etc/hosts\r\n+ sed -i \'1i172.21.15.122 smithi122.front.sepia.ceph.com smithi122\\\' /etc/hosts\r\n+ break\r\n+ command -v zypper\r\n+ command -v apt-get\r\n+ \'[\' -e /.cephlab_rc_local \']\'\r\n+ exit 0\r\n\r\x1b[K[\x1b[0;32m OK \x1b[0m] Started /etc/rc.d/rc.local Compatibility.\r\r\n Starting Hold until boot process finishes up...\r\r\n Starting Terminate Plymouth Boot Screen...\r\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n' 2020-11-16T22:26:45.950 DEBUG:teuthology.orchestra.console:expect after: b'smithi122 login: ' 2020-11-16T22:26:45.953 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:45.953 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:45.966+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:45.953 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:26:45.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:45.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:45.954 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:45.954 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:45.954 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:45.955 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:45.955 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:46.052 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.066+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.087 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.088 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.097 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:46.103 INFO:teuthology.misc:Re-opening connections... 2020-11-16T22:26:46.103 INFO:teuthology.misc:trying to connect to ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:26:46.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:26:46.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:26:46.153 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:46.167+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:46.253 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.267+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.262 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.262 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.271 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:46.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:46.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:46.436 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.436 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.445 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:46.454 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.468+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.499 INFO:teuthology.orchestra.run.smithi122:> true 2020-11-16T22:26:46.555 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.569+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.580 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.581 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.590 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:46.656 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.670+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~147s 2020-11-16T22:26:46.713 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.713 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.722 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:46.757 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.771+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.846 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.846 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.856 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:46.857 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:46.871+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:46.958 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:46.972+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:46.981 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:46.981 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:46.990 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:47.059 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:47.073+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:47.130 DEBUG:teuthology.misc:waited 1.02677321434021 2020-11-16T22:26:47.155 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.155 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.160 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:47.174+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:47.164 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:47.261 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:47.275+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:47.289 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.290 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.298 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:47.362 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:47.376+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:47.425 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.426 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.434 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:47.477 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:47.491+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:47.559 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.559 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.564 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:47.578+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:47.568 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:47.665 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:47.679+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:47.732 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.732 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.741 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:47.784 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:47.798+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:47.866 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:47.881+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:47.869 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:47.869 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:47.878 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:47.956 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:26:47.956 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:47.967 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:47.981+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:48.044 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:48.044 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:48.053 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:48.068 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:48.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:48.111 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:48.131 DEBUG:tasks.ceph_manager:Mounting data for osd.2 on ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:26:48.131 INFO:tasks.ceph_manager:Mounting osd.2: dev: ubuntu@smithi122.front.sepia.ceph.com, cluster: cephmountpoint: /var/lib/ceph/osd/ceph-2, type: xfs, options: ['noatime'] 2020-11-16T22:26:48.132 INFO:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2020-11-16T22:26:48.132 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:48.133 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:48.133 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:48.133 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:48.133 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:48.134 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:48.135 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:48.169 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:48.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:48.181 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:48.181 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:48.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:48.181 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68580177999999, 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.182 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:48.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:48.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:48.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:48.192 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:48.193 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68568957299999, 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:48.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:48.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:48.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:48.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:48.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.204 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68559933500001, 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:48.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:48.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:48.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:48.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68548269199999, 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.216 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68533432300001, 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.238 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68514495400001, 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:48.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:48.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:48.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68493103200001, 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:48.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:48.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:48.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.260 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.684732131, 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:48.261 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:48.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:48.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:48.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68455089400001, 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.271 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:48.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:48.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:48.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:48.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68392358, 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.281 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:48.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:48.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:48.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:48.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.683626237, 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.291 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:48.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:48.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:48.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:48.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.683219425, 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.301 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:48.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:48.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.683120207, 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.309 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:48.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:48.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68292524099999, 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:48.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:48.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:48.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:48.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.328 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68236416900001, 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:48.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:48.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:48.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.335 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68220568800001, 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.336 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:48.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:48.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:48.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.682072777, 2020-11-16T22:26:48.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:48.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:48.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.350 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 210.68190416799999, 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:48.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 191.44620040699999, 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.358 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:48.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:48.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:48.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 191.44607338, 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:48.369 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:48.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:48.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:48.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:48.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:48.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:48.380 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:48.382 INFO:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:26:48.383 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:48.383 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:48.396+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:48.422 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T22:26:48.423 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T22:26:48.464 INFO:tasks.ceph.osd.2:Started 2020-11-16T22:26:48.465 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:26:48.482 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:48.496+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:48.514 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:48.514 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:48.514 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:48.514 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:48.523 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:48.583 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:48.597+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:48.647 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:48.647 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:48.647 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:48.647 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:48.648 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:48.656 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:48.683 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:48.697+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:48.781 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:48.782 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:48.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:48.782 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:48.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.782 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 218.70620862999999, 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:48.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:48.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 218.59477894899999, 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.791 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:48.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:48.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:48.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28634239799999, 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.799 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:48.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:48.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:48.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28622709800001, 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:48.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:48.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:48.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:48.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28611833799999, 2020-11-16T22:26:48.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:48.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:48.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:48.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:48.828 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.286035989, 2020-11-16T22:26:48.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:48.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.839 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28594818100001, 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:48.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:48.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:48.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:48.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.849 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28585364200001, 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:48.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:48.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:48.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:48.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:48.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:48.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:48.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.860 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.285767456, 2020-11-16T22:26:48.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:48.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:48.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:48.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:48.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.871 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28566707100001, 2020-11-16T22:26:48.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:48.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:48.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:48.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:48.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:48.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28055375, 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:48.883 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:48.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:48.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28044757699999, 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.892 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:48.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:48.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.28027356699999, 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.901 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:48.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:48.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:48.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:48.909 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.280177288, 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:48.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:48.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:48.918 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.279989586, 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:48.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:48.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.925 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.279797632, 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.926 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:48.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:48.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:48.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:48.934 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.279635068, 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:48.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:48.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:48.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:48.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:48.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:48.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.944 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:48.946 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.27945920600001, 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.947 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:48.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:48.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:48.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:48.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:48.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:48.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:48.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:48.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:48.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:48.962 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.962 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.279291161, 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.964 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:48.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:48.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.972 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.279050031, 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:48.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:48.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:48.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:48.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:48.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:48.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.981 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:48.981 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:48.981 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:48.982 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:26:48.983 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:48.996+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:48.983 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:49.083 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:49.098+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:49.120 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:49.121 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:49.128 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2020-11-16T22:26:49.130 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:49.131 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-16T22:26:49.131 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2020-11-16T22:26:49.184 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.198+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:49.256 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:49.257 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:49.257 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:49.257 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:49.257 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:49.265 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:49.306 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.320+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:49.386 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.400+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:49.432 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:49.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93747289199999, 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:49.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:49.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:49.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:49.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93736068499999, 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.445 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:49.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:49.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:49.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:49.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.937270447, 2020-11-16T22:26:49.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:49.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:49.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:49.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:49.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:49.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93715380399999, 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:49.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:49.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.937005435, 2020-11-16T22:26:49.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:49.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:49.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:49.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:49.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.490 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93681606600001, 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:49.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:49.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:49.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:49.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93660214400001, 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.502 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:49.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:49.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:49.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.513 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.936403243, 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:49.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:49.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:49.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:49.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.523 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93622200600001, 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:49.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:49.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:49.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.935594692, 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.534 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:49.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:49.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:49.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.543 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.935297349, 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:49.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:49.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:49.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:49.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.553 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.934890537, 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:49.554 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:49.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:49.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.934791319, 2020-11-16T22:26:49.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:49.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:49.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:49.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:49.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:49.571 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:49.572 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93459635299999, 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:49.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:49.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:49.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:49.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.582 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93403528100001, 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93387680000001, 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:49.590 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:49.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.596 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.933743889, 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:49.597 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:49.604 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 211.93357528000001, 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:49.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:49.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:49.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.611 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 192.69787151899999, 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.612 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:49.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:49.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:49.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:49.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:49.622 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 192.697744492, 2020-11-16T22:26:49.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:49.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:49.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:49.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:49.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:49.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:49.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:49.634 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:49.636 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:49.636 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.649+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:49.736 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.750+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:49.768 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:49.768 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:49.777 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:49.807 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:49.822+0000 7f5cb1014ec0 -1 Falling back to public interface 2020-11-16T22:26:49.837 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:49.851+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:49.901 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:49.901 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:49.902 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:49.902 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:49.902 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:49.910 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:49.938 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:49.952+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:50.038 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:50.052+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:50.077 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:50.078 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.00191935000001, 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:50.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:50.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:50.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.086 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 219.890489669, 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:50.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:50.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:50.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.094 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.582053118, 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:50.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:50.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:50.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:50.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.104 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.581937818, 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:50.105 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:50.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:50.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:50.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.114 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.58182905800001, 2020-11-16T22:26:50.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:50.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.58174670899999, 2020-11-16T22:26:50.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:50.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:50.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:50.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:50.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:50.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:50.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.338 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.581658901, 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:50.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:50.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:50.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.348 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.58156436199999, 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:50.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:50.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:50.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:50.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.358 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.58147817599999, 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:50.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:50.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.368 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.58137779099999, 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57626447000001, 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57615829700001, 2020-11-16T22:26:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57598428700001, 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57588800799999, 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57570030599999, 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57550835200001, 2020-11-16T22:26:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57534578799999, 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.575169926, 2020-11-16T22:26:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57500188099999, 2020-11-16T22:26:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.57476075100001, 2020-11-16T22:26:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:50.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:50.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:50.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:50.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:50.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.861 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:50.862 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:50.862 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:50.862 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:50.864 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:50.865 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:50.878+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:50.964 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:50.978+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:50.989 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:26:51.002 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:51.002 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:51.002 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:51.002 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:51.012 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:51.017 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:51.038 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:51.039 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:51.039 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:51.039 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:26:51.039 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:26:51.041 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:26:51.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:26:51.095 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:51.109+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:51.166 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.180+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.177 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.177 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.186 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:51.270 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:51.284+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:51.351 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.351 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.360 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:51.365 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:51.379+0000 7f5cb1014ec0 -1 osd.2 28 log_to_monitors {default=true} 2020-11-16T22:26:51.404 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.417+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.469 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:51.483+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:51.496 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.496 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.506 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:51.589 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.602+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.670 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.670 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.671 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:51.685+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:51.679 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:51.771 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.784+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.843 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.843 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.852 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:51.893 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.906+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.973 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:51.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:51.979 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:51.979 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:51.989 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:51.990 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:26:52.074 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.087+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:52.113 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.113 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.122 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:52.174 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:52.188+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:52.247 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.247 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.257 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:52.298 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:52.312+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:52.376 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:52.391+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:52.382 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.382 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.391 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:52.477 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.490+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:52.516 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.516 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.525 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:52.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2020-11-16T22:26:52.578 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:52.593+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:52.679 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.692+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:52.689 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.690 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.699 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:52.780 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.793+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:52.827 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.827 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.836 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:52.880 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.894+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:52.959 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:52.960 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:52.969 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:52.980 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:52.994+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:53.042 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:53.081 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:53.095+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:53.096 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:53.096 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:53.105 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:53.182 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:53.195+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:53.191 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:53.210 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:53.231 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:26:53.232 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:53.233 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:53.233 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:26:53.233 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:53.233 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:53.233 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:53.234 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:53.242 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:53.283 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:53.296+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:53.383 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:53.396+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:53.406 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:53.407 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:53.415 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:53.484 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:53.498+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:53.581 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:53.582 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.50605629, 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:53.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:53.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:53.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.394626609, 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.591 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:53.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:53.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:53.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.086190058, 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:53.599 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:53.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:53.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:53.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:53.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.608 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.086074758, 2020-11-16T22:26:53.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:53.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:53.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08596599800001, 2020-11-16T22:26:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08588364900001, 2020-11-16T22:26:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08579584099999, 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08570130199999, 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08561511600001, 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08551473099999, 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:53.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.679 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08040141000001, 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:53.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:53.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:53.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.687 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.080295237, 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:53.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:53.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:53.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08012122700001, 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:53.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.08002494799999, 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.704 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.07983724600001, 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:53.712 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:53.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:53.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.718 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.07964529200001, 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:53.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:53.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:53.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:53.726 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.07948272799999, 2020-11-16T22:26:53.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:53.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:53.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:53.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:53.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:53.735 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.079306866, 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:53.743 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.07913882099999, 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:53.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:53.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:53.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:53.751 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.07889769100001, 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:53.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:53.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:53.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:53.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:53.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:53.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:53.759 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:53.761 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:53.761 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:53.775+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:53.861 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:53.875+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:53.891 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:53.891 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:53.891 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:53.891 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:53.900 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:53.962 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:53.976+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:54.003 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:26:54.025 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:54.025 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:54.025 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:54.025 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:54.026 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:54.034 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:54.075 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:54.089+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:54.132 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:26:54.164 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:54.177+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:54.202 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:54.202 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:54.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:54.202 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.706850554, 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.203 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:54.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:54.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:54.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:54.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.214 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.706738347, 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:54.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:54.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:54.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:54.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:54.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70664810900001, 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.226 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:54.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:54.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:54.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:54.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:54.236 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:54.237 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.706531466, 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:54.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:54.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:54.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:54.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70638309700001, 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:54.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:54.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:54.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:54.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:54.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:54.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:54.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:54.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:54.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:54.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70619372799999, 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.262 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:54.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:54.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:54.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:54.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:54.272 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70597980599999, 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:54.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:54.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:54.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:54.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:54.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70578090500001, 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.285 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:54.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:54.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70559966799999, 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.295 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:54.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70497235400001, 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:54.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:54.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70467501100001, 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:54.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.324 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70426819900001, 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:54.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:54.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:54.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.332 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70416898100001, 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:54.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:54.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:54.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:54.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.342 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.703974015, 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:54.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:54.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.352 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70341294299999, 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:54.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:54.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:54.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70325446199999, 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:54.360 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:54.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:54.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.70312155100001, 2020-11-16T22:26:54.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:54.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.374 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.702952942, 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:54.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:54.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:54.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 197.467249181, 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.382 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:54.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:54.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:54.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:54.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.392 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 197.46712215400001, 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:54.393 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:54.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:54.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:54.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:54.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:54.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:54.404 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:54.406 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:54.406 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:26:54.406 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:26:54.406 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:54.407 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:54.419+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:54.407 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:54.507 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:54.520+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:54.540 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:54.540 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:54.540 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:54.540 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:54.549 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:54.607 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:54.622+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:54.708 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:54.723+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:54.713 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:54.713 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:54.713 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:26:54.713 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:26:54.714 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:54.722 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:54.809 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:54.824+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:54.848 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:54.849 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 224.77289041099999, 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:54.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:54.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:54.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.857 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 224.66146072999999, 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:54.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:54.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:54.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35302417899999, 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.866 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:54.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:54.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:54.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:54.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35290887900001, 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:54.876 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.885 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35280011899999, 2020-11-16T22:26:54.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:54.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:54.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:54.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:54.895 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35271777, 2020-11-16T22:26:54.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:54.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:54.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:54.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:54.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:54.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.906 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35262996200001, 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:54.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:54.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.916 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35253542300001, 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:54.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:54.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.926 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.352449237, 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:54.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:54.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:54.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:54.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.35234885200001, 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:54.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:54.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:54.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:54.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.347235531, 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:54.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.34712935799999, 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.955 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:54.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:54.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:54.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:54.962 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:54.963 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.34695534799999, 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:54.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:54.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:54.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:54.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:54.971 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.346859069, 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:54.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.979 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.346671367, 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:54.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:54.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:54.985 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:54.986 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.346479413, 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:54.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:54.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:54.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:54.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:54.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.994 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.346316849, 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:54.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:54.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:54.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:55.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:55.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.002 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.34614098700001, 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:55.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:55.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:55.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:55.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.345972942, 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:55.011 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:55.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:55.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:55.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:55.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.345731812, 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:55.019 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:55.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:55.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:55.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.026 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:55.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:55.027 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:55.027 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:55.028 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:26:55.029 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:55.042+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:55.029 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:26:55.129 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:55.142+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:55.158 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:26:55.159 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:55.168 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:26:55.229 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:55.243+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:55.244 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:55.292 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:55.292 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:26:55.292 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:26:55.292 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:26:55.292 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:55.301 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:26:55.330 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:55.344+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:55.389 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:55.409 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:55.424 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:26:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:26:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [] 2020-11-16T22:26:55.425 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:55.428 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:26:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:26:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:26:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:26:55.431 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:55.444+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:55.434 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:55.530 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:55.544+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:55.570 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:55.570 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:55.579 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:55.631 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:55.645+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:55.732 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:55.745+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:55.744 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:55.744 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:55.753 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:55.833 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:55.847+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:55.877 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:55.877 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:55.886 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:55.934 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:55.948+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:56.022 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.022 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.032 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:56.034 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.047+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.135 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:56.149+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:56.156 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.157 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.166 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:56.236 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:56.250+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:56.290 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.290 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.300 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:26:56.336 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.350+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.436 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.436 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.438 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.451+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.445 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:26:56.539 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.553+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.571 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.571 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.580 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:26:56.640 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.654+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.704 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.704 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.713 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:56.757 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:56.771+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:56.841 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:56.855+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:56.850 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:56.850 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:56.859 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:56.943 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:56.957+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:57.024 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:57.024 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:57.030 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:26:57.033 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:57.074 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:57.088+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:57.144 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:57.158+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:57.198 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:57.198 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:57.207 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:57.252 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:57.265+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:57.344 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:57.344 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:57.346 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:57.361+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:57.353 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:57.441 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:26:57.446 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:57.459+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:57.517 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:26:57.518 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:26:57.527 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:26:57.547 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:57.561+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:57.581 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:26:57.601 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:26:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:26:57.622 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:26:57.622 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:26:57.623 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:26:57.648 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:57.663+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:57.693 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:57.693 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:26:57.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 227.61773049999999, 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:26:57.694 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:26:57.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.701 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 227.50630081899999, 2020-11-16T22:26:57.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:26:57.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:26:57.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:26:57.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:57.710 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19786426799999, 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:57.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:26:57.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:26:57.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:26:57.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:26:57.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:57.720 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19774896800001, 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:57.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:26:57.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:26:57.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:26:57.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:26:57.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:57.730 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.197640208, 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:57.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:26:57.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:26:57.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:26:57.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:26:57.739 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:57.740 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.197557859, 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:57.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:26:57.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:26:57.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:26:57.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:26:57.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:57.750 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19747005100001, 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:26:57.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:26:57.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:26:57.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:26:57.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:57.760 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19737551200001, 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:26:57.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:26:57.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:26:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:26:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.197289326, 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:26:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:26:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19718894100001, 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:26:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:26:57.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:26:57.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:57.790 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19207562, 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:26:57.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:26:57.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:26:57.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:57.798 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19196944699999, 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:26:57.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:26:57.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.191795437, 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:26:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:26:57.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:26:57.814 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19169915800001, 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:57.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:26:57.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:57.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:26:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:26:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.191511456, 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.023 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:26:58.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:26:58.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.029 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.191319502, 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:26:58.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:26:58.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:26:58.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.037 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19115693800001, 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:26:58.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:26:58.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:26:58.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.19098107600001, 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:26:58.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:26:58.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.190813031, 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:26:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:26:58.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 220.190571901, 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:26:58.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:26:58.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:26:58.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:26:58.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:26:58.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:26:58.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:26:58.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:26:58.270 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:58.272 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:26:58.272 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:58.285+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:58.273 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:26:58.372 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:58.386+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:58.415 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:58.415 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:26:58.415 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:26:58.415 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:58.425 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:26:58.473 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:58.487+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:26:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:26:58.551 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:58.559 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:26:58.574 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:58.588+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:58.675 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:58.688+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [] 2020-11-16T22:26:58.725 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:26:58.734 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:26:58.776 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:58.789+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:58.871 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:58.871 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:58.871 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:26:58.871 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:58.876 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:58.890+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:58.881 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:26:58.977 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:58.991+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:59.046 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:59.047 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:26:59.047 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:26:59.047 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:26:59.047 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:59.056 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:26:59.097 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:59.111+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:59.179 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:26:59.193+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:26:59.222 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72681966799999, 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:26:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:26:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:26:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:26:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:26:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:26:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72670746099999, 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:26:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:26:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:26:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:26:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:26:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72661722300001, 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:26:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:26:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:26:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:26:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:26:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72650057999999, 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:26:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:26:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:26:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:26:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:26:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72635221100001, 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:26:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:26:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:26:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:26:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72616284200001, 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:26:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:26:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:26:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:26:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:26:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72594892000001, 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:26:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.725750019, 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:26:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:26:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:26:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:26:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72556878200001, 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:26:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:26:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:26:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:26:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.724941468, 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:26:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:26:59.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:26:59.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:26:59.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.724644125, 2020-11-16T22:26:59.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:59.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:26:59.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:26:59.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:26:59.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:26:59.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:59.345 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.724237313, 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:26:59.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.353 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.724138095, 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:26:59.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:26:59.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:26:59.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.363 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72394312899999, 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:26:59.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:26:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:26:59.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:26:59.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72338205700001, 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:26:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:26:59.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:26:59.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72322357600001, 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.381 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:26:59.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:26:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:26:59.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.723090665, 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:26:59.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 221.72292205599999, 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:26:59.396 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:26:59.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.402 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 202.48721829499999, 2020-11-16T22:26:59.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:26:59.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:26:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:26:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:26:59.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:26:59.414 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 202.487091268, 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:26:59.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:26:59.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:26:59.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:26:59.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:26:59.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:26:59.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:26:59.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:26:59.426 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:26:59.426 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:26:59.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:26:59.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:26:59.427 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:26:59.428 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:26:59.429 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:59.440+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:59.528 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:26:59.542+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:26:59.565 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:26:59.565 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:26:59.565 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:26:59.566 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:26:59.574 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:26:59.628 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:59.643+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:26:59.830 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:26:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:26:59.833 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:26:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2020-11-16T22:26:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:26:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:26:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:26:58.021168+0000", 2020-11-16T22:26:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:26:51.993936+0000", 2020-11-16T22:26:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:26:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:26:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5801984, 2020-11-16T22:26:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "noscrub", 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:26:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:26:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "jewel", 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:26:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:26:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:26:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:26:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:26:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:26:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:26:59.843 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:26:59.844 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:26:59.845 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:26:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:26:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:26:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:26:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:26:59.850 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:26:59.851 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:26:59.852 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:26:59.853 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:26:59.854 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:26:59.854 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:26:59.854 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:00.058 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:00.058 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:00.058 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:00.059 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:00.060 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.061 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:00.062 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:00.063 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:00.064 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:27:00.065 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:00.066 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:00.067 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:00.068 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:00.069 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:27:00.071 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:27:00.072 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:00.073 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:00.274 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:00.274 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:00.274 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:00.274 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:27:00.276 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:00.277 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:00.278 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:00.279 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:00.280 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:27:00.281 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:00.282 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:27:00.283 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 63, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.285 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.287 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.288 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.289 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.490 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:00.490 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.491 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:27:00.492 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:27:00.494 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 29, 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:27:00.496 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.497 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.499 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.500 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:27:00.501 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:27:00.503 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:00.504 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 8, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 28, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 63, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 63, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 29, 2020-11-16T22:27:00.505 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.507 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.510 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:00.511 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:00.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:00.714 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:27:00.714 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/6963", 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/6963", 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/6963", 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/6963", 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:27:00.716 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:00.717 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:27:00.719 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:00.720 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 29 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [], 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:27:00.721 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [], 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:27:00.722 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:27:00.723 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:27:00.724 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:27:00.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:00.926 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:27:00.927 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:00.927 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:00.928 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:00.928 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:26:59.846+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:00.929 INFO:tasks.thrashosds.thrasher:j is {'epoch': 66, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:26:58.021168+0000', 'last_up_change': '2020-11-16T22:26:51.993936+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 8, 'last_clean_end': 28, 'up_from': 63, 'up_thru': 63, 'down_at': 29, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 6963}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 6963}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 6963}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 6963}]}, 'public_addr': '172.21.15.122:6801/6963', 'cluster_addr': '172.21.15.122:6803/6963', 'heartbeat_back_addr': '172.21.15.122:6807/6963', 'heartbeat_front_addr': '172.21.15.122:6805/6963', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 29}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:27:00.929 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:27:00.930 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:27:00.932 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:00.945+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:01.031 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:01.045+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:01.069 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:01.069 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:01.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:01.069 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 230.993535189, 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.070 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:01.077 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 230.882105508, 2020-11-16T22:27:01.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:01.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:01.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:01.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:01.085 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:01.086 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.573668957, 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:01.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:01.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:01.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.096 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57355365699999, 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:01.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:01.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:01.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.573444897, 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:01.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:01.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:01.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:01.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.116 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57336254800001, 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:01.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:01.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:01.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:01.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:01.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.328 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57327473999999, 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.329 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:01.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:01.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:01.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:01.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.338 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57318020100001, 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.339 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:01.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57309401500001, 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.57299363000001, 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.567880309, 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.567774136, 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.567600126, 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56750384700001, 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:01.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:01.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.604 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56731614500001, 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.605 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:01.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:01.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.567124191, 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:01.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56696162700001, 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56678576499999, 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.628 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:01.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:01.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.838 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:01.838 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:01.838 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56661772000001, 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:01.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:01.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:01.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.56637659, 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:01.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:01.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:01.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:01.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:01.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:01.854 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:01.855 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:01.857 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:01.858 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:01.858 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:27:01.858 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:01.858 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:01.858 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:27:01.859 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:27:01.859 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2020-11-16T22:26:59.679149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5495,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":17876,"kb_used_data":1660,"kb_used_omap":26,"kb_used_meta":16165,"kb_avail":283097644,"statfs":{"total":289910292480,"available":289891987456,"internally_reserved":0,"allocated":1699840,"data_stored":790025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27325,"internal_metadata":16553283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.679552"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"24","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008850+0000","last_change":"2020-11-16T22:26:53.008216+0000","last_active":"2020-11-16T22:26:53.008850+0000","last_peered":"2020-11-16T22:26:53.008850+0000","last_clean":"2020-11-16T22:26:53.008850+0000","last_became_active":"2020-11-16T22:26:53.007571+0000","last_became_peered":"2020-11-16T22:26:53.007571+0000","last_unstale":"2020-11-16T22:26:53.008850+0000","last_undegraded":"2020-11-16T22:26:53.008850+0000","last_fullsized":"2020-11-16T22:26:53.008850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"61","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012872+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:26:55.012872+0000","last_peered":"2020-11-16T22:26:55.012872+0000","last_clean":"2020-11-16T22:26:55.012872+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:26:55.012872+0000","last_undegraded":"2020-11-16T22:26:55.012872+0000","last_fullsized":"2020-11-16T22:26:55.012872+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"24","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.014241+0000","last_change":"2020-11-16T22:26:53.013570+0000","last_active":"2020-11-16T22:26:53.014241+0000","last_peered":"2020-11-16T22:26:53.014241+0000","last_clean":"2020-11-16T22:26:53.014241+0000","last_became_active":"2020-11-16T22:26:53.013036+0000","last_became_peered":"2020-11-16T22:26:53.013036+0000","last_unstale":"2020-11-16T22:26:53.014241+0000","last_undegraded":"2020-11-16T22:26:53.014241+0000","last_fullsized":"2020-11-16T22:26:53.014241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"58","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012680+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:26:55.012680+0000","last_peered":"2020-11-16T22:26:55.012680+0000","last_clean":"2020-11-16T22:26:55.012680+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:26:55.012680+0000","last_undegraded":"2020-11-16T22:26:55.012680+0000","last_fullsized":"2020-11-16T22:26:55.012680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"21","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.013958+0000","last_change":"2020-11-16T22:26:53.013292+0000","last_active":"2020-11-16T22:26:53.013958+0000","last_peered":"2020-11-16T22:26:53.013958+0000","last_clean":"2020-11-16T22:26:53.013958+0000","last_became_active":"2020-11-16T22:26:53.012784+0000","last_became_peered":"2020-11-16T22:26:53.012784+0000","last_unstale":"2020-11-16T22:26:53.013958+0000","last_undegraded":"2020-11-16T22:26:53.013958+0000","last_fullsized":"2020-11-16T22:26:53.013958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"23","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.013970+0000","last_change":"2020-11-16T22:26:53.013545+0000","last_active":"2020-11-16T22:26:53.013970+0000","last_peered":"2020-11-16T22:26:53.013970+0000","last_clean":"2020-11-16T22:26:53.013970+0000","last_became_active":"2020-11-16T22:26:53.013133+0000","last_became_peered":"2020-11-16T22:26:53.013133+0000","last_unstale":"2020-11-16T22:26:53.013970+0000","last_undegraded":"2020-11-16T22:26:53.013970+0000","last_fullsized":"2020-11-16T22:26:53.013970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"58","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012649+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:26:55.012649+0000","last_peered":"2020-11-16T22:26:55.012649+0000","last_clean":"2020-11-16T22:26:55.012649+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:26:55.012649+0000","last_undegraded":"2020-11-16T22:26:55.012649+0000","last_fullsized":"2020-11-16T22:26:55.012649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"61","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012516+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:26:55.012516+0000","last_peered":"2020-11-16T22:26:55.012516+0000","last_clean":"2020-11-16T22:26:55.012516+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:26:55.012516+0000","last_undegraded":"2020-11-16T22:26:55.012516+0000","last_fullsized":"2020-11-16T22:26:55.012516+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"32","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.009123+0000","last_change":"2020-11-16T22:26:53.008484+0000","last_active":"2020-11-16T22:26:53.009123+0000","last_peered":"2020-11-16T22:26:53.009123+0000","last_clean":"2020-11-16T22:26:53.009123+0000","last_became_active":"2020-11-16T22:26:53.007745+0000","last_became_peered":"2020-11-16T22:26:53.007745+0000","last_unstale":"2020-11-16T22:26:53.009123+0000","last_undegraded":"2020-11-16T22:26:53.009123+0000","last_fullsized":"2020-11-16T22:26:53.009123+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"21","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008884+0000","last_change":"2020-11-16T22:26:53.008366+0000","last_active":"2020-11-16T22:26:53.008884+0000","last_peered":"2020-11-16T22:26:53.008884+0000","last_clean":"2020-11-16T22:26:53.008884+0000","last_became_active":"2020-11-16T22:26:53.007629+0000","last_became_peered":"2020-11-16T22:26:53.007629+0000","last_unstale":"2020-11-16T22:26:53.008884+0000","last_undegraded":"2020-11-16T22:26:53.008884+0000","last_fullsized":"2020-11-16T22:26:53.008884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"21","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.013721+0000","last_change":"2020-11-16T22:26:53.013228+0000","last_active":"2020-11-16T22:26:53.013721+0000","last_peered":"2020-11-16T22:26:53.013721+0000","last_clean":"2020-11-16T22:26:53.013721+0000","last_became_active":"2020-11-16T22:26:53.012801+0000","last_became_peered":"2020-11-16T22:26:53.012801+0000","last_unstale":"2020-11-16T22:26:53.013721+0000","last_undegraded":"2020-11-16T22:26:53.013721+0000","last_fullsized":"2020-11-16T22:26:53.013721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"25","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008838+0000","last_change":"2020-11-16T22:26:53.008150+0000","last_active":"2020-11-16T22:26:53.008838+0000","last_peered":"2020-11-16T22:26:53.008838+0000","last_clean":"2020-11-16T22:26:53.008838+0000","last_became_active":"2020-11-16T22:26:53.007556+0000","last_became_peered":"2020-11-16T22:26:53.007556+0000","last_unstale":"2020-11-16T22:26:53.008838+0000","last_undegraded":"2020-11-16T22:26:53.008838+0000","last_fullsized":"2020-11-16T22:26:53.008838+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"60","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012514+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:26:55.012514+0000","last_peered":"2020-11-16T22:26:55.012514+0000","last_clean":"2020-11-16T22:26:55.012514+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:26:55.012514+0000","last_undegraded":"2020-11-16T22:26:55.012514+0000","last_fullsized":"2020-11-16T22:26:55.012514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"53","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012740+0000","last_change":"2020-11-16T22:23:49.863990+0000","last_active":"2020-11-16T22:26:55.012740+0000","last_peered":"2020-11-16T22:26:55.012740+0000","last_clean":"2020-11-16T22:26:55.012740+0000","last_became_active":"2020-11-16T22:23:49.863602+0000","last_became_peered":"2020-11-16T22:23:49.863602+0000","last_unstale":"2020-11-16T22:26:55.012740+0000","last_undegraded":"2020-11-16T22:26:55.012740+0000","last_fullsized":"2020-11-16T22:26:55.012740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"21","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008435+0000","last_change":"2020-11-16T22:26:53.006351+0000","last_active":"2020-11-16T22:26:53.008435+0000","last_peered":"2020-11-16T22:26:53.008435+0000","last_clean":"2020-11-16T22:26:53.008435+0000","last_became_active":"2020-11-16T22:26:53.005919+0000","last_became_peered":"2020-11-16T22:26:53.005919+0000","last_unstale":"2020-11-16T22:26:53.008435+0000","last_undegraded":"2020-11-16T22:26:53.008435+0000","last_fullsized":"2020-11-16T22:26:53.008435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"72","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012576+0000","last_change":"2020-11-16T22:26:53.005211+0000","last_active":"2020-11-16T22:26:55.012576+0000","last_peered":"2020-11-16T22:26:55.012576+0000","last_clean":"2020-11-16T22:26:55.012576+0000","last_became_active":"2020-11-16T22:26:53.005002+0000","last_became_peered":"2020-11-16T22:26:53.005002+0000","last_unstale":"2020-11-16T22:26:55.012576+0000","last_undegraded":"2020-11-16T22:26:55.012576+0000","last_fullsized":"2020-11-16T22:26:55.012576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"24","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008881+0000","last_change":"2020-11-16T22:26:53.008286+0000","last_active":"2020-11-16T22:26:53.008881+0000","last_peered":"2020-11-16T22:26:53.008881+0000","last_clean":"2020-11-16T22:26:53.008881+0000","last_became_active":"2020-11-16T22:26:53.007655+0000","last_became_peered":"2020-11-16T22:26:53.007655+0000","last_unstale":"2020-11-16T22:26:53.008881+0000","last_undegraded":"2020-11-16T22:26:53.008881+0000","last_fullsized":"2020-11-16T22:26:53.008881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"53","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012698+0000","last_change":"2020-11-16T22:23:49.863801+0000","last_active":"2020-11-16T22:26:55.012698+0000","last_peered":"2020-11-16T22:26:55.012698+0000","last_clean":"2020-11-16T22:26:55.012698+0000","last_became_active":"2020-11-16T22:23:49.863415+0000","last_became_peered":"2020-11-16T22:23:49.863415+0000","last_unstale":"2020-11-16T22:26:55.012698+0000","last_undegraded":"2020-11-16T22:26:55.012698+0000","last_fullsized":"2020-11-16T22:26:55.012698+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"19'11","reported_seq":"70","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012802+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:26:55.012802+0000","last_peered":"2020-11-16T22:26:55.012802+0000","last_clean":"2020-11-16T22:26:55.012802+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:26:55.012802+0000","last_undegraded":"2020-11-16T22:26:55.012802+0000","last_fullsized":"2020-11-16T22:26:55.012802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4726,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"71","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012510+0000","last_change":"2020-11-16T22:26:53.009437+0000","last_active":"2020-11-16T22:26:55.012510+0000","last_peered":"2020-11-16T22:26:55.012510+0000","last_clean":"2020-11-16T22:26:55.012510+0000","last_became_active":"2020-11-16T22:26:53.009082+0000","last_became_peered":"2020-11-16T22:26:53.009082+0000","last_unstale":"2020-11-16T22:26:55.012510+0000","last_undegraded":"2020-11-16T22:26:55.012510+0000","last_fullsized":"2020-11-16T22:26:55.012510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"65","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012999+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:26:55.012999+0000","last_peered":"2020-11-16T22:26:55.012999+0000","last_clean":"2020-11-16T22:26:55.012999+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:26:55.012999+0000","last_undegraded":"2020-11-16T22:26:55.012999+0000","last_fullsized":"2020-11-16T22:26:55.012999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"61","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012696+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:26:55.012696+0000","last_peered":"2020-11-16T22:26:55.012696+0000","last_clean":"2020-11-16T22:26:55.012696+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:26:55.012696+0000","last_undegraded":"2020-11-16T22:26:55.012696+0000","last_fullsized":"2020-11-16T22:26:55.012696+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"24","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008179+0000","last_change":"2020-11-16T22:26:53.006653+0000","last_active":"2020-11-16T22:26:53.008179+0000","last_peered":"2020-11-16T22:26:53.008179+0000","last_clean":"2020-11-16T22:26:53.008179+0000","last_became_active":"2020-11-16T22:26:53.006191+0000","last_became_peered":"2020-11-16T22:26:53.006191+0000","last_unstale":"2020-11-16T22:26:53.008179+0000","last_undegraded":"2020-11-16T22:26:53.008179+0000","last_fullsized":"2020-11-16T22:26:53.008179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"21","reported_epoch":"64","state":"active+clean+wait","last_fresh":"2020-11-16T22:26:53.008064+0000","last_change":"2020-11-16T22:26:53.006609+0000","last_active":"2020-11-16T22:26:53.008064+0000","last_peered":"2020-11-16T22:26:53.008064+0000","last_clean":"2020-11-16T22:26:53.008064+0000","last_became_active":"2020-11-16T22:26:53.006166+0000","last_became_peered":"2020-11-16T22:26:53.006166+0000","last_unstale":"2020-11-16T22:26:53.008064+0000","last_undegraded":"2020-11-16T22:26:53.008064+0000","last_fullsized":"2020-11-16T22:26:53.008064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"19'5","reported_seq":"77","reported_epoch":"65","state":"active+clean","last_fresh":"2020-11-16T22:26:55.012811+0000","last_change":"2020-11-16T22:26:53.009677+0000","last_active":"2020-11-16T22:26:55.012811+0000","last_peered":"2020-11-16T22:26:55.012811+0000","last_clean":"2020-11-16T22:26:55.012811+0000","last_became_active":"2020-11-16T22:26:53.009228+0000","last_became_peered":"2020-11-16T22:26:53.009228+0000","last_unstale":"2020-11-16T22:26:55.012811+0000","last_undegraded":"2020-11-16T22:26:55.012811+0000","last_fullsized":"2020-11-16T22:26:55.012811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":5495,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":10990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10534,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":63,"seq":270582939650,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5860,"kb_used_data":540,"kb_used_omap":26,"kb_used_meta":5285,"kb_avail":94365980,"statfs":{"total":96636764160,"available":96630763520,"internally_reserved":0,"allocated":552960,"data_stored":257247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27325,"internal_metadata":5412163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738416,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6012,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365828,"statfs":{"total":96636764160,"available":96630607872,"internally_reserved":0,"allocated":577536,"data_stored":266197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:26:15 2020","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":1.028,"15min":1.028},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96299999999999997,"5min":1.052,"15min":1.052},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738416,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6004,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365836,"statfs":{"total":96636764160,"available":96630616064,"internally_reserved":0,"allocated":569344,"data_stored":266581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:26:15 2020","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.434,"15min":0.434},"max":{"1min":0.95299999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]}]}} 2020-11-16T22:27:02.470 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:27:02.471 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [2, 0, 1] 2020-11-16T22:27:02.472 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '2', '0', '1'] 2020-11-16T22:27:02.472 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 2 0 1 2020-11-16T22:27:02.473 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:01.869+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:02.473 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:27:02.475 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:27:02.475 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:02.489+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:02.576 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:02.590+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:02.610 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:02.610 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:02.619 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:27:02.676 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:02.689+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:02.745 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:02.746 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:02.756 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:27:02.797 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:02.810+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:02.832 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: you must specify pairs of osd ids to be remapped 2020-11-16T22:27:02.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-16T22:27:02.835 INFO:tasks.thrashosds.thrasher:Failed to rm-pg-upmap-items, ignoring 2020-11-16T22:27:02.878 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:02.892+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:02.920 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:02.920 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:02.929 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:02.979 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:02.993+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:03.067 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.067 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.077 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:03.081 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:03.095+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:03.181 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:03.195+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:03.201 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.201 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.210 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:03.282 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:03.296+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:03.375 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.375 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.383 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:03.397+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:03.385 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:03.485 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:03.499+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:03.523 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.523 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.532 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:03.586 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:03.599+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:03.687 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:03.701+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:03.696 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.697 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.706 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:03.788 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:03.802+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:03.832 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.832 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.841 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:27:03.859 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:03.888 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:03.902+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:03.972 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:03.972 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:03.981 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:27:04.010 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:04.022 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:04.035+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:04.029 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:04.052 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:27:04.053 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:04.053 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:04.053 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:27:04.090 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:04.104+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:04.145 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:27:04.145 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:27:04.145 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:27:04.146 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:27:04.146 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:27:04.154 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:27:04.191 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:04.205+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:04.278 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:27:04.278 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:27:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:27:04.279 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [] 2020-11-16T22:27:04.279 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:27:04.287 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:04.292 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:04.305+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:04.393 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:04.407+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:04.420 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:04.420 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:04.420 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:04.421 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:04.429 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:04.477 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:27:04.493 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:04.507+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:04.593 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:04.594 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:04.594 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:04.594 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:04.594 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:04.595 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:04.608+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:04.602 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:04.695 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:04.709+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:04.728 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:04.728 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:04.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.233063294, 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:04.729 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:04.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:04.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:04.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:04.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:04.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:04.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:04.740 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.232951087, 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:04.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:04.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:04.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:04.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:04.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:04.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:04.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23286084899999, 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:04.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:04.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:04.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:04.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:04.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23274420600001, 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:04.763 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:04.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:04.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:04.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:04.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23259583699999, 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:04.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:04.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:04.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.984 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:04.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:04.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:04.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:05.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:05.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:05.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.191 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23240646799999, 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:05.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:05.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:05.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:05.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:05.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:05.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:05.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:05.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:05.406 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23219254599999, 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:05.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:05.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:05.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:05.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:05.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:05.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:05.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:05.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:05.620 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23199364499999, 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:06.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.035 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.231812408, 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:06.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:06.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:06.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.045 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23118509400001, 2020-11-16T22:27:06.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:06.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:06.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:06.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.055 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23088775100001, 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:06.056 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:06.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:06.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:06.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23048093899999, 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.270 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:06.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:06.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23038172099999, 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:06.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:06.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:06.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:06.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:06.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:06.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.23018675500001, 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.288 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:06.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:06.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:06.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.501 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.22962568299999, 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:06.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.710 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.229467202, 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:06.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.717 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.22933429099999, 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:06.718 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:06.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:06.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:06.926 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 227.229165682, 2020-11-16T22:27:06.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:06.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:06.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:06.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.934 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 207.99346192100001, 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:06.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:06.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:06.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:06.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:07.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:07.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:07.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:07.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.147 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 207.99333489399999, 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:07.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:07.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:07.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:07.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:07.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:07.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:07.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:07.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:07.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:07.361 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:07.361 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:07.361 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:07.363 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:27:07.364 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:07.376+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:07.364 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:07.463 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:07.476+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:07.501 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:07.501 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:07.501 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:07.502 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:07.511 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:07.563 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:07.577+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:07.575 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:07.595 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:27:07.596 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:07.596 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:07.596 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:27:07.636 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:07.636 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:07.646 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:07.663 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:27:07.677+0000 7f5ca882c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 6958) UID: 0 2020-11-16T22:27:07.764 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:07.778+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:07.811 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:07.811 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:07.820 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:27:07.836 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:27:07.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:07.836 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2] 2020-11-16T22:27:07.836 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.2 doing powercycle of ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:27:07.836 INFO:teuthology.orchestra.console:Power off smithi122 2020-11-16T22:27:07.837 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:27:07.862 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:27:07.874 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:07.888+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:07.955 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:07.956 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:07.965 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:27:07.974 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:07.988+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:08.075 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:08.089+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:08.090 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.090 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.099 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:27:08.176 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.189+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.223 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.224 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.233 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:08.276 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.289+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.370 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.370 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.377 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.390+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.379 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:08.478 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.491+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.545 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.545 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.554 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:08.595 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.609+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.679 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:08.694+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:08.719 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.719 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.729 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:08.781 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:08.794+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:08.862 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.862 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:08.871 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:08.915 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:08.929+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:08.982 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:08.996+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:08.996 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:08.996 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.005 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:09.083 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:09.097+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:09.170 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:09.171 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.180 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:09.184 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:09.198+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:09.284 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:09.297+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:09.309 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:09.310 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.319 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:09.365 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:27:09.385 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:09.399+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:09.483 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:09.483 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.486 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:09.501+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:09.492 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:09.588 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:09.601+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:09.607 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:09.617 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:09.617 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.626 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:09.689 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:09.703+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:09.749 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:09.753 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:09.754 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:09.763 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:09.769 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:09.788 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:27:09.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:09.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:09.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:27:09.791 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:09.804+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:09.891 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:09.905+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:09.926 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:09.927 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:09.927 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:09.927 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:09.927 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:09.935 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:09.993 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:10.007+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:10.062 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:10.062 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:10.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:10.062 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:10.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 239.98603328300001, 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:10.063 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:10.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:10.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:10.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:10.070 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 239.87460360200001, 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:10.071 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:10.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:10.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:10.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:10.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56616705100001, 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:10.079 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:10.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:10.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:10.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:10.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:10.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:10.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:10.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:10.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.290 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.566051751, 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:10.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:10.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:10.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:10.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:10.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:10.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:10.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:10.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:10.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:10.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.503 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:10.706 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:10.706 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56594299099999, 2020-11-16T22:27:10.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:10.706 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:10.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:10.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:10.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:10.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:10.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:10.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:10.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:10.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:10.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:10.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:10.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:10.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:11.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.122 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:11.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56586064199999, 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:11.123 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:11.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:11.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:11.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:11.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:11.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:11.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:11.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:11.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:11.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.565772834, 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:11.540 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:11.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:11.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:11.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:11.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.565678295, 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:11.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:11.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:11.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:11.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:11.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:11.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:11.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:11.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:11.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:12.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:12.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:12.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:12.168 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56559210899999, 2020-11-16T22:27:12.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:12.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:12.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:12.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:12.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:12.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:12.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:12.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:12.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:12.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:12.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:12.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:12.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:12.585 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.565491724, 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:12.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:12.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:12.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:12.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:12.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:12.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:12.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:12.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:12.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:12.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:12.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56037840299999, 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:13.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:13.003 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:13.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:13.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:13.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:13.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:13.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:13.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:13.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:13.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:13.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:13.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:13.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:13.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:13.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:13.416 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56027223000001, 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:13.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:13.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:13.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:13.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:13.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:13.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:13.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:13.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:13.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:13.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:13.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:13.627 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.56009821999999, 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:14.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:14.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:14.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:14.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:14.041 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.560001941, 2020-11-16T22:27:14.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:14.043 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:14.043 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:14.043 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:14.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:14.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:14.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:14.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:14.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:14.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:14.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:14.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:14.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:14.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:14.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:14.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.456 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.55981423899999, 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:14.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:14.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:14.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:14.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.55962228499999, 2020-11-16T22:27:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:14.870 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:14.870 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:14.870 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:14.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:14.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:14.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:14.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:14.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:14.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:14.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:15.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:15.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:15.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:15.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:15.080 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.559459721, 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:15.081 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:15.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:15.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:15.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:15.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:15.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:15.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:15.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:15.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.559283859, 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:15.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:15.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:15.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:15.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:15.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:15.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:15.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:15.707 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:15.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:15.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:15.707 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:15.707 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:15.910 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.55911581399999, 2020-11-16T22:27:16.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:16.114 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:16.114 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:16.114 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:16.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:16.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:16.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:16.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:16.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:16.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:16.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:16.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:16.324 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.55887468399999, 2020-11-16T22:27:16.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:16.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:16.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:16.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:16.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:16.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:16.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:16.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:16.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:16.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:16.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:16.536 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:16.537 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:16.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:27:16.550 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:27:16.551 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:16.551+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:16.552 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:16.639 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:16.653+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:16.689 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:16.689 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:16.689 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:16.689 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:16.698 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:16.737 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:16.741 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:16.755+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:16.750 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2020-11-16T22:27:16.758 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:16.758 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:16.758 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:16.758 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:16.759 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:16.760 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:16.761 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:16.824 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:16.824 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:16.833 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:16.874 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:16.888+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:16.941 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:16.955+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:16.998 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:16.998 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.007 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:17.042 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:17.055+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:17.142 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:17.156+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:17.144 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.145 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.154 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:17.243 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:17.257+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:17.319 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.320 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.328 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:17.344 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:17.358+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:17.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2020-11-16T22:27:17.445 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:17.459+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:17.493 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.493 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.502 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:17.545 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:17.560+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:17.630 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.630 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.639 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:17.682 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:17.696+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:17.747 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:17.761+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:17.764 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.764 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.773 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:17.857 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:17.871+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:17.938 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:17.939 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:17.947 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:17.949 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:17.963+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.049 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:18.063+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.074 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.074 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.083 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:18.151 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:18.165+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:18.248 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.248 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.251 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:18.265+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:18.257 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:18.352 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:18.366+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.381 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.382 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.390 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:18.453 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:18.467+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.519 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.519 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.528 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:18.552 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:27:18.572 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:18.586+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:18.653 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.653 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.655 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:18.669+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:18.663 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:18.756 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:18.770+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:18.762 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:18.828 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:18.828 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:18.837 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:18.881 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:18.895+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.912 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:18.933 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:18.933 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:18.934 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:18.934 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:18.934 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:18.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:18.936 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:18.957 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:18.972+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:18.971 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:18.971 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:18.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:18.971 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:18.980 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:19.059 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:19.073+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:19.115 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:27:19.146 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:19.147 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:19.147 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:19.147 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:19.147 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:19.156 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:19.200 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:19.214+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:19.261 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:19.275+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:19.282 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:19.282 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:19.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 249.20670504099999, 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:19.283 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:19.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:19.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:19.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 249.09527535999999, 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:19.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:19.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:19.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78683880899999, 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.305 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:19.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:19.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.314 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78672350900001, 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:19.315 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:19.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:19.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:19.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:19.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:19.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:19.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78661474899999, 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:19.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:19.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.7865324, 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:19.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.546 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78644459200001, 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:19.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:19.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:19.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:19.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:19.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78635005300001, 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:19.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:19.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:19.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:19.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.786263867, 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:19.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:19.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78616348200001, 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:19.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:19.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:19.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:19.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:19.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:19.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:19.993 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.781050161, 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:19.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:19.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:19.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:19.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:19.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.001 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.78094398799999, 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:20.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:20.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:20.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.780769978, 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.010 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:20.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:20.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:20.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:20.221 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.780673699, 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:20.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:20.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:20.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.229 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.780485997, 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:20.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:20.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:20.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:20.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.438 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.780294043, 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:20.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:20.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:20.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.780131479, 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:20.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.77995561700001, 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.455 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:20.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:20.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.779787572, 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.463 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:20.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:20.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:20.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.470 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 241.779546442, 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:20.471 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:20.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:20.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:20.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.478 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:20.479 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:20.479 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:20.480 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:20.494+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:20.481 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:20.582 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:20.596+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:20.617 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:20.617 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:20.618 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:20.618 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:20.627 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:20.682 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:20.696+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:20.752 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:20.761 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:20.804 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:20.818+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:20.883 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:20.898+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:20.887 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:20.887 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:20.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:20.887 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:20.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:20.888 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:20.888 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:20.888 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.39184710200001, 2020-11-16T22:27:20.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:24.994 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:24.994 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:24.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:24.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:24.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:24.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:25.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:25.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:25.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:25.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.005 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.39173489500001, 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:25.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:25.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:25.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:25.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:25.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:25.016 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.391644657, 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:25.017 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:25.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:25.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:25.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:25.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:25.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:25.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:25.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:25.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:25.435 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.435 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:25.638 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.638 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.39152801399999, 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:25.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:25.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:25.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:25.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:25.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:25.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:25.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:25.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:25.852 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.391379645, 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:25.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:25.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:25.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:25.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:25.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:26.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:26.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:26.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:26.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:26.066 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.391190276, 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:26.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:26.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:26.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:26.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.279 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.390976354, 2020-11-16T22:27:26.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:26.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:26.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:26.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:26.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:26.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:26.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:26.493 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:26.494 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.390777453, 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:26.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:26.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:26.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:26.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:26.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.39059621600001, 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.38996890199999, 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.717 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:26.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:26.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:26.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:26.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.38967155899999, 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.727 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:26.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:26.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:26.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:26.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.736 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.389264747, 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:26.737 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:26.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.389165529, 2020-11-16T22:27:26.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:26.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:26.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:26.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:26.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:26.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:26.755 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.38897056299999, 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.388409491, 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:26.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:26.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.772 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.38825101, 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.773 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:26.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:26.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:26.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.780 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.388118099, 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:26.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:26.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:26.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.38794949000001, 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:26.788 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:26.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:26.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:26.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:26.795 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 224.15224572899999, 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:26.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:26.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 224.152118702, 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:26.807 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:26.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:26.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:26.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:26.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:26.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:26.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:26.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:26.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:26.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.817 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.818 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.818 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:26.818 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:26.818 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:26.818 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:26.819 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:27:26.820 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:26.820 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:26.820 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:27:26.820 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:27:26.821 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:27:26.821 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:27:26.822 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:26.823 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:26.834+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:26.921 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:26.935+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:26.962 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:26.962 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:27:26.963 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:26.964 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:26.964 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:26.964 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:26.964 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:26.975 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:27.021 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:27.041 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:27.042 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:27.042 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:27.042 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:27.042 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:27.044 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:27.044 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:27.057 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.072+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.123 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:27.137+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:27.139 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.139 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.148 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:27.231 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.245+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.313 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.314 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.323 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:27.365 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.379+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.426 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:27.439+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:27.461 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.461 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.470 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:27.554 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:27.568+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:27.627 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:27.641+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:27.629 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2020-11-16T22:27:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.645 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:27.728 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.743+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.810 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.810 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.819 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:27.863 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.877+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.887 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:27:27.929 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:27.943+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:27.946 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:27.947 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:27.956 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:28.040 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:28.054+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:28.120 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.121 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.129 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:28.170 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:28.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:28.232 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:28.246+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:28.293 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.293 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.302 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:28.346 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:28.360+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:28.433 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.433 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.433 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:28.447+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:28.442 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:28.533 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:28.548+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:28.606 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.607 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.615 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:28.659 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:28.673+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:28.735 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:28.750+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:28.740 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.740 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.749 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:28.836 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:28.850+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:28.876 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:28.876 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:28.885 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:28.937 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:28.951+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:29.010 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:29.010 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:29.019 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:29.037 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:29.051+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:29.046 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:29.138 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:29.152+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:29.183 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:29.184 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:29.192 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:29.239 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:29.253+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:29.250 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:29.250 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:29.251 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:29.251 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:29.251 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:29.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:29.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:29.321 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:29.321 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:29.321 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:29.321 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:29.330 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:29.340 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:29.354+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:29.441 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:29.456+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:29.454 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:29.454 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:29.463 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:29.542 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:29.556+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.589 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09374647000001, 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:29.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:29.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:29.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.600 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09363426300001, 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:29.601 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:29.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:29.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.093544025, 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.613 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:29.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:29.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:29.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:29.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:29.623 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09342738199999, 2020-11-16T22:27:29.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:29.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:29.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:29.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:29.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:29.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.635 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.093279013, 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:29.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:29.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:29.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:29.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:29.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.093089644, 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:29.647 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:29.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:29.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:29.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:29.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:29.658 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.092875722, 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:29.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:29.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:29.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:29.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.092676821, 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:29.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09249558400001, 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.680 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:29.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:29.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09186826999999, 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.690 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:29.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:29.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09157092699999, 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.700 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:29.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.091164115, 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.710 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:29.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:29.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.091064897, 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.718 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:29.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:29.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:29.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:29.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.09086993099999, 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.728 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:29.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:29.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:29.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.090308859, 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.738 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:29.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:29.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:29.744 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:29.745 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.090150378, 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:29.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:29.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:29.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.752 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.090017467, 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:29.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:29.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.759 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.08984885800001, 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.760 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:29.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:29.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:29.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:29.767 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 232.85414509700001, 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:29.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:29.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:29.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:29.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:29.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.778 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 232.85401807, 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:29.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:29.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:29.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:29.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:29.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:29.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.789 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:29.790 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:29.790 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:29.791 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:29.792 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:29.805+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:29.892 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:29.905+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:29.902 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:27:29.919 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:29.920 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:29.920 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:29.920 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:29.928 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:29.992 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:30.007+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:30.054 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:30.054 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:30.054 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:30.054 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:30.054 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:30.063 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:30.104 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:30.118+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:30.194 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:30.208+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:30.229 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:30.229 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:30.230 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 260.15386316899998, 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:30.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:30.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:30.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:30.238 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 260.04243348799997, 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:30.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:30.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:30.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.246 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.733996937, 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:30.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:30.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:30.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:30.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.733881637, 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:30.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73377287700001, 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.267 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:30.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:30.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73369052800001, 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.277 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:30.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:30.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:30.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:30.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:30.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:30.289 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:30.290 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73360271999999, 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:30.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:30.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:30.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:30.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:30.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73350818099999, 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.301 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:30.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:30.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:30.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.310 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73342199499999, 2020-11-16T22:27:30.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:30.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:30.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:30.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:30.320 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.73332160999999, 2020-11-16T22:27:30.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:30.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:30.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:30.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:30.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:30.330 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72820828900001, 2020-11-16T22:27:30.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:30.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:30.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:30.339 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.728102116, 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:30.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:30.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:30.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.347 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72792810600001, 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:30.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:30.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:30.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.355 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72783182699999, 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:30.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:30.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:30.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72764412500001, 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:30.364 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:30.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:30.370 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72745217100001, 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:30.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:30.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:30.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.378 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72728960699999, 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:30.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:30.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:30.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.386 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.727113745, 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:30.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:30.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:30.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72694569999999, 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.395 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.72670457000001, 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.403 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:30.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:30.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:30.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:30.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:30.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:30.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:30.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:30.411 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:30.412 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:30.426+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:30.413 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:30.513 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:30.527+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:30.541 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:30.542 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:30.542 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:30.542 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:30.551 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:30.614 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:30.628+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:30.714 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:30.728+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:30.716 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:30.716 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:30.716 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:30.716 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:30.716 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:30.725 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:30.815 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:30.829+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:30.893 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:30.893 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:30.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:30.893 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39792984100001, 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.894 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.905 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39781763400001, 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:30.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:30.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:30.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:30.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:30.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39772739599999, 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.917 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:30.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:30.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:30.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:30.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:30.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:30.928 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39761075300001, 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:30.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39746238399999, 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.940 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:30.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:30.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:30.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:30.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:30.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:30.950 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.397273015, 2020-11-16T22:27:30.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:30.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:30.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.962 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.397059093, 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:30.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:30.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:30.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.973 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39686019199999, 2020-11-16T22:27:30.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:30.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.983 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.396678955, 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:30.984 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:30.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:30.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.993 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39605164100001, 2020-11-16T22:27:30.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:30.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:30.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:30.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:30.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.003 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39575429800001, 2020-11-16T22:27:31.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:31.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:31.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.013 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39534748599999, 2020-11-16T22:27:31.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:31.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:31.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:31.021 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39524826799999, 2020-11-16T22:27:31.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:31.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:31.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:31.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:31.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:31.031 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:31.032 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39505330200001, 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:31.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:31.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:31.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:31.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:31.041 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39449223, 2020-11-16T22:27:31.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:31.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:31.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:31.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.049 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.394333749, 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:31.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:31.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:31.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.056 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.39420083799999, 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.057 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:31.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:31.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:31.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:31.064 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.394032229, 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:31.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:31.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:31.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 234.15832846800001, 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:31.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:31.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:31.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:31.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:31.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 234.15820144099999, 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:31.083 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:31.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:31.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:31.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:31.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:31.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:31.093 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.094 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.094 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.094 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:31.094 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:31.094 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:31.095 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:27:31.096 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:31.097 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:31.109+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:31.195 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:31.210+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:31.228 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:31.228 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:31.229 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:31.229 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:31.237 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:31.255 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:31.296 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:31.311+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:31.361 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:31.361 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:31.361 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:31.362 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:31.362 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:31.370 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:31.397 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:31.411+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:31.419 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:31.439 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:31.439 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:31.440 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:31.440 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:31.442 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:31.443 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:31.495 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:31.495 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:31.497 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:31.512+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:31.505 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:31.598 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:31.612+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:31.636 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:31.636 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:31.646 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:31.699 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:31.714+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:31.800 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:31.815+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:31.810 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:31.811 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:31.819 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:31.900 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:31.914+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:31.945 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:31.945 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:31.954 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:31.973 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:31.974 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:31.974 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:27:31.974 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:27:31.974 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:27:31.974 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:27:32.040 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:32.054+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:32.085 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.085 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.095 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:32.103 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.117+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.125 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:32.126 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:27:32.126 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:32.203 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:32.218+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:32.250 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.251 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.260 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:32.304 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.318+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.405 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.419+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.426 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.426 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.435 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:32.505 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.519+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.563 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.563 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.572 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:32.606 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:32.620+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:32.706 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.720+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.737 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.738 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.747 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:32.830 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.844+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.908 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:32.922+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:32.912 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:32.912 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:32.921 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:33.009 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:33.023+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:33.047 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:33.047 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:33.056 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:33.096 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:27:33.110 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:33.124+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:33.181 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:33.182 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:33.190 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:33.232 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:33.246+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:33.312 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:33.326+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:33.355 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:33.356 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:33.365 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:33.413 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:33.426+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:33.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2020-11-16T22:27:33.444 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:33.493 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:33.494 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:33.502 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:33.543 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:33.557+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:33.607 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:33.614 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:33.628+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:33.627 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:33.631 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:33.631 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:33.669 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:33.669 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:33.669 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:33.669 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:33.669 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:33.678 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:33.715 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:33.730+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:33.815 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:33.830+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:33.845 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.846 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.35053333600001, 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:33.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:33.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:33.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:33.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:33.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.35042112899998, 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.858 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:33.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:33.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:33.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:33.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:33.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.868 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.350330891, 2020-11-16T22:27:33.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:33.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:33.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:33.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.880 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.35021424799999, 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:33.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:33.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:33.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.350065879, 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.892 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:33.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:33.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:33.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.902 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34987651, 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:33.903 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:33.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:33.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:33.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:33.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:33.913 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:33.914 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.349662588, 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:33.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.916 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34946368700002, 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:33.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:33.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.930 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:33.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:33.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.935 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34928244999998, 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.936 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:33.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:33.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.945 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34865513599999, 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:33.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:33.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:33.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.955 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34835779299999, 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:33.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:33.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:33.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:33.965 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.347950981, 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:33.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:33.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:33.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.973 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34785176299999, 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:33.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:33.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:33.983 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34765679700001, 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.984 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:33.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.987 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:33.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:33.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:33.992 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:33.993 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34709572499997, 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:33.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:33.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:33.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:33.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.000 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.346937244, 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:34.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:34.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:34.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:34.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34680433300002, 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:34.008 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:34.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:34.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:34.014 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:34.015 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 256.34663572400001, 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:34.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:34.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 237.11093196300001, 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:34.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:34.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:34.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:34.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.033 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 237.11080493599999, 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:34.034 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:34.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:34.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:34.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:34.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:34.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:34.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.044 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:34.045 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:27:34.047 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:34.047 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:34.060+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:34.147 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:34.161+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:34.176 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:34.176 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:34.176 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:34.176 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:34.185 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:34.248 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:34.262+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:34.348 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:34.362+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:34.348 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:34.349 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:34.349 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:34.349 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:34.349 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:34.359 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:34.449 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:34.464+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:34.525 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 264.45015181100001, 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 264.33872213000001, 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.03028557900001, 2020-11-16T22:27:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:34.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:34.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:34.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:34.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:34.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:34.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:35.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:35.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:35.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:35.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:35.161 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.030170279, 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:35.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:35.163 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:35.163 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:35.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:35.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:35.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:35.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:35.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:35.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.578 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:35.579 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:35.579 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.03006151900001, 2020-11-16T22:27:35.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:35.579 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:35.782 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:35.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:35.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:35.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:35.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:35.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:35.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:35.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:35.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:35.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:35.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:35.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:35.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:35.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:35.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:35.995 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:36.198 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:36.198 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02997916999999, 2020-11-16T22:27:36.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:36.198 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:36.198 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:36.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:36.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:36.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:36.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:36.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:36.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:36.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:36.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:36.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:36.614 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:36.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.029891362, 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:36.615 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:36.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:36.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:36.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:36.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:36.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:36.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:36.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:36.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:36.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:36.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:36.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.030 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02979682300003, 2020-11-16T22:27:37.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:37.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:37.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:37.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:37.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:37.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:37.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:37.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:37.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:37.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:37.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02971063699999, 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:37.448 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:37.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:37.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:37.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:37.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:37.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:37.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:37.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:37.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:37.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:37.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.029610252, 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:38.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:38.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:38.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:38.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02449693099999, 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.077 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:38.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:38.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:38.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:38.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:38.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:38.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:38.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02439075799998, 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:38.499 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02421674800001, 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:38.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:38.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:38.507 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02412046900002, 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:38.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:38.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:38.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.515 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02393276700002, 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:38.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:38.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:38.521 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:38.522 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02374081300002, 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:38.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:38.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:38.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:38.530 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02357824900002, 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02340238699998, 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02323434200002, 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:38.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:38.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:38.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:38.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.756 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 257.02299321200002, 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:38.757 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:38.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:38.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:38.765 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:38.766 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:38.766 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:27:38.767 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:38.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:38.767 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:27:38.769 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:38.780+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:38.770 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:38.867 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:38.880+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:38.908 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:38.908 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:38.908 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:38.908 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:38.917 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:38.967 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:38.981+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:38.997 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:39.035 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:39.039 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:39.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:39.041 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:39.042 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:39.051 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:39.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:39.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:39.170 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:39.183+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:39.183 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:39.234 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:27:39.235 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:27:39.235 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2020-11-16T22:27:39.235 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:27:39.235 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:27:39.235 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:27:33.929220+0000", 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:27:12.077974+0000", 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5799936, 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:27:39.236 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:27:39.237 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "jewel", 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:27:39.238 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:27:39.241 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:39.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:39.243 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:39.244 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:39.245 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.246 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:27:39.249 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:39.250 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:27:39.251 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:39.252 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:39.254 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:39.255 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:39.256 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:39.257 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:27:39.258 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:27:39.259 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:39.260 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:27:39.261 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:39.262 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:39.263 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:39.264 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:39.265 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:39.266 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:39.267 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:39.268 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:27:39.269 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:27:39.270 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:27:39.271 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:27:39.272 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:27:39.273 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:27:39.274 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:27:39.275 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:27:39.276 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:27:39.277 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:27:39.278 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:27:39.279 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.280 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:39.281 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 70, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:39.282 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.284 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:39.286 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.287 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:39.288 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.289 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:27:39.290 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0, 2020-11-16T22:27:39.292 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 70, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:39.293 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.294 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.296 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.297 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.298 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.299 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:27:39.300 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:27:39.301 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 0, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 8, 2020-11-16T22:27:39.303 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 28, 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 63, 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 63, 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 70, 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.304 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:27:39.305 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.306 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:27:39.307 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:27:39.310 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 6963 2020-11-16T22:27:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/6963", 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/6963", 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/6963", 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/6963", 2020-11-16T22:27:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "exists" 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:27:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:27:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:27:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 70 2020-11-16T22:27:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [], 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [], 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:27:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:27:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:27:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:27:39.522 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:27:39.522 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:27:39.522 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:27:39.523 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:27:39.523 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:27:39.523 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:27:39.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:27:39.523 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:27:39.525 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:39.525 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:39.526 INFO:tasks.thrashosds.thrasher:j is {'epoch': 75, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:27:33.929220+0000', 'last_up_change': '2020-11-16T22:27:12.077974+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 70, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 70, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 8, 'last_clean_end': 28, 'up_from': 63, 'up_thru': 63, 'down_at': 70, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 6963}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 6963}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 6963}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 6963}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 6963}]}, 'public_addr': '172.21.15.122:6801/6963', 'cluster_addr': '172.21.15.122:6803/6963', 'heartbeat_back_addr': '172.21.15.122:6807/6963', 'heartbeat_front_addr': '172.21.15.122:6805/6963', 'state': ['exists']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 70}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:27:39.527 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:27:39.527 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:39.538+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:39.527 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:39.625 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:39.639+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:39.652 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:39.652 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:39.661 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:39.726 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:39.739+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:39.786 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:39.786 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:39.795 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:39.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2020-11-16T22:27:39.836 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:39.849+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:39.858 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:27:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:27:39.870 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2020-11-16T22:27:39.686116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5495,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":35,"acting":35,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":18300,"kb_used_data":1828,"kb_used_omap":26,"kb_used_meta":16421,"kb_avail":283097220,"statfs":{"total":289910292480,"available":289891553280,"internally_reserved":0,"allocated":1871872,"data_stored":879570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27325,"internal_metadata":16815427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.772740"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"64","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915114+0000","last_change":"2020-11-16T22:27:13.096544+0000","last_active":"2020-11-16T22:27:30.915114+0000","last_peered":"2020-11-16T22:27:30.915114+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:27:13.096544+0000","last_became_peered":"2020-11-16T22:27:13.096544+0000","last_unstale":"2020-11-16T22:27:30.915114+0000","last_undegraded":"2020-11-16T22:27:30.915114+0000","last_fullsized":"2020-11-16T22:27:13.094918+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"70","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.912046+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:27:30.912046+0000","last_peered":"2020-11-16T22:27:30.912046+0000","last_clean":"2020-11-16T22:27:30.912046+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:27:30.912046+0000","last_undegraded":"2020-11-16T22:27:30.912046+0000","last_fullsized":"2020-11-16T22:27:30.912046+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"16'6","reported_seq":"60","reported_epoch":"74","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:27:30.912022+0000","last_change":"2020-11-16T22:27:13.096173+0000","last_active":"2020-11-16T22:27:30.912022+0000","last_peered":"2020-11-16T22:27:30.912022+0000","last_clean":"2020-11-16T22:23:17.513148+0000","last_became_active":"2020-11-16T22:27:13.096173+0000","last_became_peered":"2020-11-16T22:27:13.096173+0000","last_unstale":"2020-11-16T22:27:30.912022+0000","last_undegraded":"2020-11-16T22:27:13.095157+0000","last_fullsized":"2020-11-16T22:27:13.095073+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"67","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911909+0000","last_change":"2020-11-16T22:23:17.515874+0000","last_active":"2020-11-16T22:27:30.911909+0000","last_peered":"2020-11-16T22:27:30.911909+0000","last_clean":"2020-11-16T22:27:30.911909+0000","last_became_active":"2020-11-16T22:23:17.515361+0000","last_became_peered":"2020-11-16T22:23:17.515361+0000","last_unstale":"2020-11-16T22:27:30.911909+0000","last_undegraded":"2020-11-16T22:27:30.911909+0000","last_fullsized":"2020-11-16T22:27:30.911909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"51","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.911869+0000","last_change":"2020-11-16T22:27:13.096117+0000","last_active":"2020-11-16T22:27:30.911869+0000","last_peered":"2020-11-16T22:27:30.911869+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:27:13.096117+0000","last_became_peered":"2020-11-16T22:27:13.096117+0000","last_unstale":"2020-11-16T22:27:30.911869+0000","last_undegraded":"2020-11-16T22:27:30.911869+0000","last_fullsized":"2020-11-16T22:27:13.094569+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"16'5","reported_seq":"59","reported_epoch":"74","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:27:30.911741+0000","last_change":"2020-11-16T22:27:13.095617+0000","last_active":"2020-11-16T22:27:30.911741+0000","last_peered":"2020-11-16T22:27:30.911741+0000","last_clean":"2020-11-16T22:23:17.506944+0000","last_became_active":"2020-11-16T22:27:13.095617+0000","last_became_peered":"2020-11-16T22:27:13.095617+0000","last_unstale":"2020-11-16T22:27:30.911741+0000","last_undegraded":"2020-11-16T22:27:13.094729+0000","last_fullsized":"2020-11-16T22:27:13.094591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"67","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.912127+0000","last_change":"2020-11-16T22:23:17.515652+0000","last_active":"2020-11-16T22:27:30.912127+0000","last_peered":"2020-11-16T22:27:30.912127+0000","last_clean":"2020-11-16T22:27:30.912127+0000","last_became_active":"2020-11-16T22:23:17.514965+0000","last_became_peered":"2020-11-16T22:23:17.514965+0000","last_unstale":"2020-11-16T22:27:30.912127+0000","last_undegraded":"2020-11-16T22:27:30.912127+0000","last_fullsized":"2020-11-16T22:27:30.912127+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"70","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911699+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:27:30.911699+0000","last_peered":"2020-11-16T22:27:30.911699+0000","last_clean":"2020-11-16T22:27:30.911699+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:27:30.911699+0000","last_undegraded":"2020-11-16T22:27:30.911699+0000","last_fullsized":"2020-11-16T22:27:30.911699+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"16'8","reported_seq":"73","reported_epoch":"74","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:27:30.915292+0000","last_change":"2020-11-16T22:27:13.095875+0000","last_active":"2020-11-16T22:27:30.915292+0000","last_peered":"2020-11-16T22:27:30.915292+0000","last_clean":"2020-11-16T22:23:45.854844+0000","last_became_active":"2020-11-16T22:27:13.095875+0000","last_became_peered":"2020-11-16T22:27:13.095875+0000","last_unstale":"2020-11-16T22:27:30.915292+0000","last_undegraded":"2020-11-16T22:27:13.094959+0000","last_fullsized":"2020-11-16T22:27:13.094825+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"51","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915224+0000","last_change":"2020-11-16T22:27:13.096681+0000","last_active":"2020-11-16T22:27:30.915224+0000","last_peered":"2020-11-16T22:27:30.915224+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:27:13.096681+0000","last_became_peered":"2020-11-16T22:27:13.096681+0000","last_unstale":"2020-11-16T22:27:30.915224+0000","last_undegraded":"2020-11-16T22:27:30.915224+0000","last_fullsized":"2020-11-16T22:27:13.095072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"51","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.911752+0000","last_change":"2020-11-16T22:27:13.096221+0000","last_active":"2020-11-16T22:27:30.911752+0000","last_peered":"2020-11-16T22:27:30.911752+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:27:13.096221+0000","last_became_peered":"2020-11-16T22:27:13.096221+0000","last_unstale":"2020-11-16T22:27:30.911752+0000","last_undegraded":"2020-11-16T22:27:30.911752+0000","last_fullsized":"2020-11-16T22:27:13.095049+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"61","reported_epoch":"74","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:27:30.915121+0000","last_change":"2020-11-16T22:27:13.097444+0000","last_active":"2020-11-16T22:27:30.915121+0000","last_peered":"2020-11-16T22:27:30.915121+0000","last_clean":"2020-11-16T22:23:10.308875+0000","last_became_active":"2020-11-16T22:27:13.097444+0000","last_became_peered":"2020-11-16T22:27:13.097444+0000","last_unstale":"2020-11-16T22:27:30.915121+0000","last_undegraded":"2020-11-16T22:27:13.095556+0000","last_fullsized":"2020-11-16T22:27:13.094827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"16'1","reported_seq":"69","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911725+0000","last_change":"2020-11-16T22:23:17.138112+0000","last_active":"2020-11-16T22:27:30.911725+0000","last_peered":"2020-11-16T22:27:30.911725+0000","last_clean":"2020-11-16T22:27:30.911725+0000","last_became_active":"2020-11-16T22:23:17.137690+0000","last_became_peered":"2020-11-16T22:23:17.137690+0000","last_unstale":"2020-11-16T22:27:30.911725+0000","last_undegraded":"2020-11-16T22:27:30.911725+0000","last_fullsized":"2020-11-16T22:27:30.911725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"62","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911857+0000","last_change":"2020-11-16T22:23:49.863990+0000","last_active":"2020-11-16T22:27:30.911857+0000","last_peered":"2020-11-16T22:27:30.911857+0000","last_clean":"2020-11-16T22:27:30.911857+0000","last_became_active":"2020-11-16T22:23:49.863602+0000","last_became_peered":"2020-11-16T22:23:49.863602+0000","last_unstale":"2020-11-16T22:27:30.911857+0000","last_undegraded":"2020-11-16T22:27:30.911857+0000","last_fullsized":"2020-11-16T22:27:30.911857+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"51","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915459+0000","last_change":"2020-11-16T22:27:13.096652+0000","last_active":"2020-11-16T22:27:30.915459+0000","last_peered":"2020-11-16T22:27:30.915459+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:27:13.096652+0000","last_became_peered":"2020-11-16T22:27:13.096652+0000","last_unstale":"2020-11-16T22:27:30.915459+0000","last_undegraded":"2020-11-16T22:27:30.915459+0000","last_fullsized":"2020-11-16T22:27:13.095292+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"86","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.911721+0000","last_change":"2020-11-16T22:27:13.095578+0000","last_active":"2020-11-16T22:27:30.911721+0000","last_peered":"2020-11-16T22:27:30.911721+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:27:13.095578+0000","last_became_peered":"2020-11-16T22:27:13.095578+0000","last_unstale":"2020-11-16T22:27:30.911721+0000","last_undegraded":"2020-11-16T22:27:30.911721+0000","last_fullsized":"2020-11-16T22:27:13.094513+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"64","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915185+0000","last_change":"2020-11-16T22:27:13.096629+0000","last_active":"2020-11-16T22:27:30.915185+0000","last_peered":"2020-11-16T22:27:30.915185+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:27:13.096629+0000","last_became_peered":"2020-11-16T22:27:13.096629+0000","last_unstale":"2020-11-16T22:27:30.915185+0000","last_undegraded":"2020-11-16T22:27:30.915185+0000","last_fullsized":"2020-11-16T22:27:13.094939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"62","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911946+0000","last_change":"2020-11-16T22:23:49.863801+0000","last_active":"2020-11-16T22:27:30.911946+0000","last_peered":"2020-11-16T22:27:30.911946+0000","last_clean":"2020-11-16T22:27:30.911946+0000","last_became_active":"2020-11-16T22:23:49.863415+0000","last_became_peered":"2020-11-16T22:23:49.863415+0000","last_unstale":"2020-11-16T22:27:30.911946+0000","last_undegraded":"2020-11-16T22:27:30.911946+0000","last_fullsized":"2020-11-16T22:27:30.911946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"19'11","reported_seq":"79","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.912037+0000","last_change":"2020-11-16T22:23:17.137034+0000","last_active":"2020-11-16T22:27:30.912037+0000","last_peered":"2020-11-16T22:27:30.912037+0000","last_clean":"2020-11-16T22:27:30.912037+0000","last_became_active":"2020-11-16T22:23:17.136578+0000","last_became_peered":"2020-11-16T22:23:17.136578+0000","last_unstale":"2020-11-16T22:27:30.912037+0000","last_undegraded":"2020-11-16T22:27:30.912037+0000","last_fullsized":"2020-11-16T22:27:30.912037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4726,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"85","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.911894+0000","last_change":"2020-11-16T22:27:13.095565+0000","last_active":"2020-11-16T22:27:30.911894+0000","last_peered":"2020-11-16T22:27:30.911894+0000","last_clean":"2020-11-16T22:27:10.078224+0000","last_became_active":"2020-11-16T22:27:13.095565+0000","last_became_peered":"2020-11-16T22:27:13.095565+0000","last_unstale":"2020-11-16T22:27:30.911894+0000","last_undegraded":"2020-11-16T22:27:30.911894+0000","last_fullsized":"2020-11-16T22:27:13.094494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'6","reported_seq":"74","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.912092+0000","last_change":"2020-11-16T22:23:17.138664+0000","last_active":"2020-11-16T22:27:30.912092+0000","last_peered":"2020-11-16T22:27:30.912092+0000","last_clean":"2020-11-16T22:27:30.912092+0000","last_became_active":"2020-11-16T22:23:17.138074+0000","last_became_peered":"2020-11-16T22:23:17.138074+0000","last_unstale":"2020-11-16T22:27:30.912092+0000","last_undegraded":"2020-11-16T22:27:30.912092+0000","last_fullsized":"2020-11-16T22:27:30.912092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"70","reported_epoch":"74","state":"active+clean","last_fresh":"2020-11-16T22:27:30.911907+0000","last_change":"2020-11-16T22:23:14.109921+0000","last_active":"2020-11-16T22:27:30.911907+0000","last_peered":"2020-11-16T22:27:30.911907+0000","last_clean":"2020-11-16T22:27:30.911907+0000","last_became_active":"2020-11-16T22:23:14.109494+0000","last_became_peered":"2020-11-16T22:23:14.109494+0000","last_unstale":"2020-11-16T22:27:30.911907+0000","last_undegraded":"2020-11-16T22:27:30.911907+0000","last_fullsized":"2020-11-16T22:27:30.911907+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"64","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915086+0000","last_change":"2020-11-16T22:27:13.096681+0000","last_active":"2020-11-16T22:27:30.915086+0000","last_peered":"2020-11-16T22:27:30.915086+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:27:13.096681+0000","last_became_peered":"2020-11-16T22:27:13.096681+0000","last_unstale":"2020-11-16T22:27:30.915086+0000","last_undegraded":"2020-11-16T22:27:30.915086+0000","last_fullsized":"2020-11-16T22:27:13.094825+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"51","reported_epoch":"74","state":"active+undersized","last_fresh":"2020-11-16T22:27:30.915400+0000","last_change":"2020-11-16T22:27:13.096621+0000","last_active":"2020-11-16T22:27:30.915400+0000","last_peered":"2020-11-16T22:27:30.915400+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:27:13.096621+0000","last_became_peered":"2020-11-16T22:27:13.096621+0000","last_unstale":"2020-11-16T22:27:30.915400+0000","last_undegraded":"2020-11-16T22:27:30.915400+0000","last_fullsized":"2020-11-16T22:27:13.095091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"19'5","reported_seq":"91","reported_epoch":"74","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:27:30.912003+0000","last_change":"2020-11-16T22:27:13.096219+0000","last_active":"2020-11-16T22:27:30.912003+0000","last_peered":"2020-11-16T22:27:30.912003+0000","last_clean":"2020-11-16T22:27:10.078586+0000","last_became_active":"2020-11-16T22:27:13.096219+0000","last_became_peered":"2020-11-16T22:27:13.096219+0000","last_unstale":"2020-11-16T22:27:30.912003+0000","last_undegraded":"2020-11-16T22:27:13.095066+0000","last_fullsized":"2020-11-16T22:27:13.094940+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":11,"acting":11,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":5495,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":10990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10534,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":11,"acting":11,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":1,"acting":1,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":63,"seq":270582939652,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5948,"kb_used_data":564,"kb_used_omap":26,"kb_used_meta":5349,"kb_avail":94365892,"statfs":{"total":96636764160,"available":96630673408,"internally_reserved":0,"allocated":577536,"data_stored":269928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27325,"internal_metadata":5477699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6148,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":94365692,"statfs":{"total":96636764160,"available":96630468608,"internally_reserved":0,"allocated":651264,"data_stored":304629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:27:19 2020","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92200000000000004,"5min":1.028,"15min":1.028},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94299999999999995,"5min":1.052,"15min":1.052},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738424,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6204,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":94365636,"statfs":{"total":96636764160,"available":96630411264,"internally_reserved":0,"allocated":643072,"data_stored":305013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:27:19 2020","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.755,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":1.097},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.501,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82199999999999995,"5min":1,"15min":1},"last":1.157}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]}]}} 2020-11-16T22:27:40.279 INFO:tasks.thrashosds.thrasher:Setting 4.7 to [2, 0] 2020-11-16T22:27:40.279 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.7', '2', '0'] 2020-11-16T22:27:40.280 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.7 2 0 2020-11-16T22:27:40.282 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:40.282 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:40.283 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:40.283 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:40.295+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:40.283 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:27:40.382 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:40.396+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:40.450 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:40.451 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:40.460 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:40.500 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:40.514+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:40.584 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:40.597+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:40.624 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:40.624 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:40.633 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:40.684 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:40.698+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:40.768 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:40.768 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:40.777 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:40.819 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:40.832+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:40.885 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:40.899+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:40.943 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:40.943 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:40.944 INFO:teuthology.orchestra.run.smithi138.stderr:set 4.7 pg_upmap mapping to [2,0] 2020-11-16T22:27:40.953 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:40.986 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:41.001+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:41.040 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:41.077 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:41.077 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:41.086 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:41.087 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:41.100+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:41.188 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:41.202+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:41.215 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:41.215 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:41.220 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:41.225 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:41.240 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:41.241 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:41.241 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:41.241 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:41.241 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:41.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:41.289 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:41.303+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:41.388 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:41.389 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:41.391 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:41.405+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:41.398 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:41.492 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:41.505+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:41.523 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.028420654, 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02830844699997, 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02821820899999, 2020-11-16T22:27:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:41.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:41.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:41.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:41.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:41.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:41.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:41.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:41.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:41.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:42.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:42.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:42.167 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02810156599998, 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:42.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:42.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:42.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:42.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:42.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:42.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:42.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:42.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:42.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:42.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:42.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:42.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:42.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:42.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:42.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:42.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.585 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02795319699999, 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:42.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:42.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:42.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:42.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:42.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:42.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:42.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:42.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:42.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:42.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:42.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:42.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:42.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:42.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:42.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:43.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:43.002 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:43.003 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:43.003 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:43.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.206 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:43.206 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:43.206 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02776382799999, 2020-11-16T22:27:43.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:43.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:43.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:43.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:43.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.624 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02754990599999, 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:43.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:43.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:43.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:43.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:43.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:43.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:43.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:43.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:43.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:44.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:44.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.043 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:44.043 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:44.043 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02735100500001, 2020-11-16T22:27:44.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:44.043 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:44.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:44.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:44.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:44.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:44.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:44.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02716976800002, 2020-11-16T22:27:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:44.662 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:44.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:44.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:44.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:44.671 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02654245399998, 2020-11-16T22:27:44.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:44.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:44.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:44.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:44.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:44.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:44.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02624511099998, 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02583829899999, 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.102 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:45.102 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02573908099998, 2020-11-16T22:27:45.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:45.102 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:45.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:45.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:45.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:45.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.111 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.025544115, 2020-11-16T22:27:45.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:45.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.323 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02498304300002, 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:45.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:45.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:45.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:45.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02482456199999, 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:45.331 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.332 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:45.332 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:45.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:45.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:45.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.540 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.02469165100001, 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:45.541 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:45.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:45.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:45.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:45.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:45.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:45.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:45.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:45.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:45.751 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 264.024523042, 2020-11-16T22:27:45.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:45.954 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:45.954 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:45.954 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:45.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:45.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:45.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:45.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:45.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:45.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:45.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:46.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:46.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:46.164 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 244.788819281, 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:46.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:46.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:46.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:46.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:46.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:46.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:46.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:46.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:46.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:46.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:46.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:46.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:46.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.582 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:46.583 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:46.583 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 244.78869225400001, 2020-11-16T22:27:46.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:46.583 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:46.786 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:46.786 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:46.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:46.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:46.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:46.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:46.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:46.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:46.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:46.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:46.996 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:46.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:46.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:46.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:47.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:47.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:47.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:47.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:47.204 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:47.204 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:27:47.205 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:47.206 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:47.206 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:47.206 INFO:tasks.thrashosds.thrasher:inject_pause on 0 2020-11-16T22:27:47.206 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:27:47.206 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:27:47.207 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2020-11-16T22:27:47.208 INFO:tasks.workunit.client.0.smithi138.stderr:Note: checking out '6441771be52a0f47b9e118f8bca335f29dfda438'. 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr:state without impacting any branches by performing another checkout. 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:27:47.209 INFO:tasks.workunit.client.0.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2020-11-16T22:27:47.210 INFO:tasks.workunit.client.0.smithi138.stderr:do so (now or later) by using -b with the checkout command again. Example: 2020-11-16T22:27:47.210 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:27:47.210 INFO:tasks.workunit.client.0.smithi138.stderr: git checkout -b 2020-11-16T22:27:47.210 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:27:47.210 INFO:tasks.workunit.client.0.smithi138.stderr:HEAD is now at 6441771be5 Merge pull request #38041 from tchaikov/wip-mon-trim-osdmap 2020-11-16T22:27:47.211 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:47.219+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:47.211 INFO:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2020-11-16T22:27:47.414 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:47.417 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:47.430+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:47.418 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:47.418 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-11-16T22:27:47.418 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:47.419 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:47.439 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:47.439 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:47.440 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:47.440 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:47.440 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:47.442 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:47.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:47.516 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:47.531+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:47.554 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:47.554 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:47.563 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:47.602 INFO:tasks.workunit.client.0.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2020-11-16T22:27:47.604 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-11-16T22:27:47.604 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2020-11-16T22:27:47.646 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:47.660+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:47.718 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:47.733+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:47.727 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:47.727 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:47.736 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:47.819 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:47.834+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:27:48.023 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:48.024 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:48.024 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:48.037+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.024 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:48.123 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:48.137+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.153 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:48.153 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:48.162 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:48.245 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:48.259+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~31s 2020-11-16T22:27:48.326 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:48.339+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:48.327 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:48.327 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:48.336 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:48.426 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:48.440+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.459 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2020-11-16T22:27:48.501 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:48.501 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:48.510 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:48.518 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2020-11-16T22:27:48.551 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:48.565+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:48.554 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-11-16T22:27:48.556 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-11-16T22:27:48.556 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2020-11-16T22:27:48.592 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-11-16T22:27:48.594 INFO:teuthology.orchestra.run.smithi138:> set -ex 2020-11-16T22:27:48.595 INFO:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-11-16T22:27:48.607 INFO:teuthology.orchestra.run.smithi138.stderr:+ dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-11-16T22:27:48.608 INFO:teuthology.orchestra.run.smithi138.stdout:rename/prepare.shrename/pri_rem.shrename/pri_nul.shrename/dir_pri_pri.shrename/pri_pri.shrename/rem_nul.shrename/dir_pri_nul.shrename/rem_pri.shrename/rem_rem.shrename/all.shcaps/mon_commands.shdirect_io/big.shdirect_io/misc.shdirect_io/test_short_dio_readdirect_io/direct_io_testdirect_io/test_sync_iokernel_untar_build.shlibcephfs/test.shcephtool/test_daemon.shcephtool/test.shcephtool/test_kvstore_tool.shmgr/test_localpool.shrgw/s3_bucket_quota.plrgw/test_rgw_orphan_list.shrgw/test_rgw_throttle.shrgw/test_rgw_gc_log.shrgw/run-s3tests.shrgw/s3_user_quota.plrgw/test_rgw_obj.shrgw/run-reshard.shrgw/test_rgw_reshard.pyrgw/s3_multipart_upload.plosdc/stress_objectcacher.shpost-file.shsuites/blogbench.shsuites/dbench.shsuites/fsx.shsuites/ffsb.shsuites/fsstress.shsuites/iogen.shsuites/cephfs_journal_tool_smoke.shsuites/fsync-tester.shsuites/dbench-short.shsuites/pjd.shsuites/bonnie.shsuites/iozone.shsuites/iozone-sync.shsuites/fio.shrados/stress_watch.shrados/test_cache_pool.shrados/clone.shrados/test_python.shrados/test_pool_access.shrados/test_crash.shrados/load-gen-mix.shrados/test_health_warnings.shrados/load-gen-mostlyread.shrados/test_pool_quota.shrados/load-gen-mix-small-long.shrados/load-gen-big.shrados/test_librados_build.shrados/test_hang.shrados/version_number_sanity.shrados/test_dedup_tool.shrados/test_envlibrados_for_rocksdb.shrados/test_large_omap_detection.pyrados/test_alloc_hint.shrados/test_rados_timeouts.shrados/load-gen-mix-small.shrados/test.shrados/test_rados_tool.shobjectstore/test_fuse.shcephadm/test_cephadm.shcephadm/create_iscsi_disks.shcephadm/test_adoption.shcephadm/test_dashboard_e2e.shcephadm/test_repos.shceph-helpers-root.sherasure-code/encode-decode-non-regression.sherasure-code/bench.shfs/test_o_truncfs/snap-hierarchy.shfs/norstats/kernel_untar_tar.shfs/test_python.shfs/misc/multiple_rsync.shfs/misc/layout_vxattrs.shfs/misc/chmod.shfs/misc/dirfrag.shfs/misc/xattrs.shfs/misc/filelock_interrupt.pyfs/misc/filelock_deadlock.pyfs/misc/rstats.shfs/misc/subvolume.shfs/misc/trivial_sync.shfs/misc/i_complete_vs_rename.shfs/misc/acl.shfs/misc/mkpool_layout_vxattrs.shfs/misc/direct_io.pyfs/quota/quota.shfs/upgrade/volume_clientfs/snaps/snaptest-double-null.shfs/snaps/snaptest-git-ceph.shfs/snaps/snaptest-2.shfs/snaps/snaptest-dir-rename.shfs/snaps/snaptest-0.shfs/snaps/snaptest-1.shfs/snaps/untar_snap_rm.shfs/snaps/snaptest-hardlink.shfs/snaps/snap-rm-diff.shfs/snaps/snaptest-authwb.shfs/snaps/snaptest-snap-rm-cmp.shfs/snaps/snaptest-intodir.shfs/snaps/snaptest-upchildrealms.shfs/snaps/snaptest-realm-split.shfs/snaps/snaptest-capwb.shfs/snaps/snaptest-parents.shfs/snaps/snaptest-xattrwb.shfs/snaps/snaptest-multiple-capsnaps.shfs/snaps/snaptest-estale.shfs/snaps/snaptest-snap-rename.shfs/test_o_trunc.shfs/multiclient_sync_read_eof.pycls/test_cls_rgw.shcls/test_cls_rbd.shcls/test_cls_hello.shcls/test_cls_sdk.shcls/test_cls_log.shcls/test_cls_lock.shcls/test_cls_cmpomap.shcls/test_cls_journal.shcls/test_cls_2pc_queue.shcls/test_cls_numops.shcls/test_cls_refcount.shrest/test_mgr_rest_api.pyrest/test-restful.shrbd/test_admin_socket.shrbd/test_rbdmap_RBDMAPFILE.shrbd/notify_master.shrbd/journal.shrbd/krbd_stable_pages_required.shrbd/rbd_mirror_snapshot.shrbd/krbd_namespaces.shrbd/krbd_udev_netlink_enobufs.shrbd/rbd_mirror_fsx_prepare.shrbd/krbd_huge_osdmap.shrbd/diff.shrbd/rbd_mirror_bootstrap.shrbd/simple_big.shrbd/krbd_fallocate.shrbd/test_librbd.shrbd/rbd_mirror_fsx_compare.shrbd/qemu-iotests.shrbd/krbd_udev_enumerate.shrbd/concurrent.shrbd/rbd-ggate.shrbd/krbd_latest_osdmap_on_map.shrbd/issue-20295.shrbd/import_export.shrbd/notify_slave.shrbd/cli_generic.shrbd/qemu_dynamic_features.shrbd/krbd_data_pool.shrbd/krbd_udev_netns.shrbd/rbd_mirror_ha.shrbd/rbd-nbd.shrbd/rbd_mirror_journal.shrbd/test_librbd_python.shrbd/read-flags.shrbd/krbd_exclusive_option.shrbd/huge-tickets.shrbd/map-snapshot-io.shrbd/image_read.shrbd/diff_continuous.shrbd/permissions.shrbd/map-unmap.shrbd/merge_diff.shrbd/test_rbd_tasks.shrbd/test_lock_fence.shrbd/rbd_mirror_helpers.shrbd/qos.shrbd/kernel.shrbd/krbd_wac.shrbd/rbd_mirror_stress.shrbd/qemu_rebuild_object_map.shrbd/krbd_udev_symlinks.shrbd/rbd_groups.shrbd/verify_pool.shrbd/test_rbd_mirror.shrestart/test-backtraces.pymon/pool_ops.shmon/test_mon_config_key.pymon/caps.shmon/rbd_snaps_ops.shmon/auth_caps.shmon/pg_autoscaler.shmon/test_config_key_caps.shmon/osd.shmon/config.shmon/ping.pymon/test_mon_osdmap_prune.shmon/crush_ops.shceph-tests/ceph-admin-commands.shtrue.shhadoop/terasort.shhadoop/repl.shhadoop/wordcount.sh 2020-11-16T22:27:48.810 INFO:teuthology.orchestra.run.smithi138.stderr:9+1 records in 2020-11-16T22:27:48.810 INFO:teuthology.orchestra.run.smithi138.stderr:9+1 records out 2020-11-16T22:27:48.810 INFO:teuthology.orchestra.run.smithi138.stderr:4744 bytes (4.7 kB, 4.6 KiB) copied, 3.6181e-05 s, 131 MB/s 2020-11-16T22:27:48.811 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2020-11-16T22:27:48.812 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2020-11-16T22:27:48.812 INFO:teuthology.orchestra.run.smithi138:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6441771be52a0f47b9e118f8bca335f29dfda438 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2020-11-16T22:27:48.814 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:48.814 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:48.815 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:48.828+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:48.815 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:48.837 INFO:tasks.workunit.client.0.smithi138.stderr:+ set -e 2020-11-16T22:27:48.837 INFO:tasks.workunit.client.0.smithi138.stderr:+ git clone git://git.ceph.com/xfstests.git 2020-11-16T22:27:48.839 INFO:tasks.workunit.client.0.smithi138.stderr:Cloning into 'xfstests'... 2020-11-16T22:27:49.043 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:49.057+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:49.143 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:49.157+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:49.169 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:49.170 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:49.179 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:49.244 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:49.258+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:49.304 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:49.304 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:49.313 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:49.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:49.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:49.439 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:49.440 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:49.443 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:49.446 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:49.461+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:49.448 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:49.548 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:49.562+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:49.574 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:49.574 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:49.583 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:49.589 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:49.608 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:49.608 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:49.609 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:49.609 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:49.609 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:49.611 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:49.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:49.648 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:49.662+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:49.710 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:49.710 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:49.711 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21542577700001, 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:49.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:49.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:49.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:49.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:49.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:49.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:50.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:50.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:50.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:50.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:50.129 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21531356999998, 2020-11-16T22:27:50.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:50.131 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:50.131 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:50.131 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:50.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:50.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:50.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:50.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:50.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:50.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:50.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:50.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:50.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:50.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:50.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21522333199999, 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:50.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:50.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:50.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:50.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:51.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21510668899998, 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:51.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:51.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:51.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:51.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:51.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21495831999999, 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.181 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:51.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:51.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:51.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:51.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:51.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.214768951, 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:51.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:51.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:51.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.214555029, 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:51.406 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21435612800002, 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:51.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.427 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21417489100003, 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:51.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:51.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:51.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:51.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:51.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:51.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.641 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21354757699999, 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.651 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21325023399999, 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:51.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:51.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:51.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:51.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21284342199999, 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:51.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.868 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:51.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.872 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21274420399999, 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.873 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:51.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:51.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:51.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:51.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21254923800001, 2020-11-16T22:27:51.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:51.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:51.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:51.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:51.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:51.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:51.893 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21198816600003, 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:51.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:51.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:51.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:51.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:51.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:51.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:52.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:52.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.211829685, 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:52.104 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:52.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.21169677400002, 2020-11-16T22:27:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:52.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:52.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:52.319 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 272.211528165, 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:52.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:52.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:52.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.97582440400001, 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:52.327 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:52.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:52.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:52.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:52.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:52.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.337 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 252.97569737699999, 2020-11-16T22:27:52.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:52.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:52.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:52.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:52.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:52.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:52.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:52.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:52.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:52.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:52.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.552 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.553 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.553 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:52.553 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:52.553 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:52.554 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:27:52.554 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:52.555 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:52.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:27:52.555 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.3814246641327821 2020-11-16T22:27:52.556 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.3814246641327821 2020-11-16T22:27:52.557 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:52.568+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:52.558 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:52.655 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:52.669+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:52.695 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:52.695 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:52.695 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:52.695 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:52.704 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:52.723 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:52.743 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:52.743 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:52.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:52.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:52.787 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:52.801+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:52.856 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:52.870+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:52.868 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:52.869 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:52.877 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:52.961 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:52.975+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:27:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 0.381425 (61a5) 2020-11-16T22:27:53.046 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.046 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.055 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:53.096 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.110+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.158 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.172+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.192 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.192 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.201 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:53.259 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.273+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.326 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.326 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.335 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:53.360 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.374+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.460 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.461 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.461 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.475+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.469 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:53.562 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:53.575+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:53.596 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.596 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.605 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:53.662 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:53.676+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:53.729 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.729 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.738 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:53.782 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.796+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.861 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.862 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:53.864 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:53.877+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:53.871 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:53.965 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:53.979+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:53.997 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:53.997 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.006 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:54.065 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:54.078+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:54.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~37s 2020-11-16T22:27:54.131 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.131 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.141 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:54.182 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:54.195+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:54.267 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.281+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:54.306 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.306 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.315 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:54.368 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.382+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:54.442 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.442 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.450 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:54.492 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.506+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:54.570 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:54.583+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:54.615 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.615 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.624 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:54.670 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.684+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:54.748 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:54.748 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.749 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.757 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:54.798 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.813+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:54.872 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:54.885+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:54.884 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:54.884 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:54.895 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:54.907 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:54.926 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:54.927 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:54.927 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:54.927 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:54.927 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:54.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:54.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:54.973 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:54.987+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:55.000 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:27:55.058 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:55.059 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:55.068 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:55.073 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:55.088+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:55.174 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:55.187+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:55.195 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:55.195 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:27:55.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:27:55.195 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:27:55.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69997304700001, 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.196 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:27:55.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:27:55.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:55.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:27:55.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.206 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69986083999999, 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:27:55.207 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:27:55.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:27:55.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:27:55.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:27:55.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:55.218 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.699770602, 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:55.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:27:55.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:27:55.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:27:55.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:27:55.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.229 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69965395899999, 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:27:55.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:27:55.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:27:55.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69950559, 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.241 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:55.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:27:55.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:27:55.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.251 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.699316221, 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:27:55.252 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:27:55.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:27:55.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:27:55.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:27:55.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:55.263 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.699102299, 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:55.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:27:55.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:27:55.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:27:55.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:27:55.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.274 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69890339800003, 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:27:55.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:27:55.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:27:55.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:27:55.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:55.284 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69872216099998, 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:27:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:27:55.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:27:55.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:27:55.293 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69809484699999, 2020-11-16T22:27:55.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:27:55.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:27:55.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.303 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.306 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69779750399999, 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.307 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:27:55.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:27:55.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:27:55.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:27:55.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.697390692, 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.317 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:27:55.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.697291474, 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.325 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:27:55.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:27:55.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.334 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69709650800002, 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:27:55.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:27:55.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:27:55.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:27:55.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.344 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69653543599998, 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:27:55.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:27:55.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:27:55.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69637695500001, 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:55.352 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:27:55.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:27:55.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.358 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69624404400003, 2020-11-16T22:27:55.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:27:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:27:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 277.69607543500001, 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:27:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:27:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:39 3.4 3:2e2fa760:::200.00000001:head [write 892~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 258.46037167399999, 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002, 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747723+0000", 2020-11-16T22:27:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4595999999999997e-05 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747777+0000", 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2029999999999998e-06 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747787+0000", 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6277999999999999e-05 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747803+0000", 2020-11-16T22:27:55.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3449999999999995e-06 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747812+0000", 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4673999999999998e-05 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747877+0000", 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5780000000000004e-06 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747883+0000", 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017944440000000001 2020-11-16T22:27:55.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749678+0000", 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6392999999999999e-05 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749744+0000", 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7545000000000002e-05 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.749842+0000", 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00082784500000000001 2020-11-16T22:27:55.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750670+0000", 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074131200000000005 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751411+0000", 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8964000000000001e-05 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751460+0000", 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4659000000000001e-05 2020-11-16T22:27:55.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751495+0000", 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037718360000000002 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 258.46024464700002, 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.385 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:27:55.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:27:55.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:27:55.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:27:55.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:27:55.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:27:55.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.395 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.396 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:27:55.396 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:27:55.396 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:55.397 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:55.411+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:55.397 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:55.497 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:55.511+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:55.524 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:55.524 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:55.524 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:27:55.524 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:55.533 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:55.598 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:55.612+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:55.656 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:55.656 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:55.657 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:55.657 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:55.657 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:55.665 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:55.709 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:55.723+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:55.791 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:55.791 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 285.716104556, 2020-11-16T22:27:55.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:55.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:55.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:55.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:55.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:55.800 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 285.604674875, 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:55.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:55.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:55.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.808 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.296238324, 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:55.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:55.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.818 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.296123024, 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:27:55.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:27:55.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:27:55.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:27:55.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:27:55.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:27:55.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:27:55.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:27:55.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:27:55.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:27:55.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:27:55.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:27:55.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:27:55.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:27:55.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:27:55.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:27:55.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:27:55.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:27:55.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.835 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29601426400001, 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:27:55.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:27:55.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:27:55.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:27:55.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29593191499998, 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.846 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:27:55.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:27:55.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:27:55.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:27:55.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.856 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29584410699999, 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.857 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:55.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:27:55.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:27:55.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:27:55.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:27:55.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.868 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29574956800002, 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.869 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:27:55.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:27:55.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:27:55.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:27:55.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:27:55.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.880 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29566338199999, 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:27:55.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:27:55.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:27:55.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:27:55.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:27:55.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:27:55.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:27:55.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:27:55.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:27:55.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:27:55.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:27:55.893 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.893 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.893 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29556299699999, 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:27:55.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:27:55.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:27:55.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:27:55.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:27:55.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:27:55.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.904 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29044967599998, 2020-11-16T22:27:55.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:55.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:27:55.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:27:55.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:27:55.913 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:55.914 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29034350299997, 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:27:55.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:27:55.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:27:55.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29016949300001, 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:55.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:27:55.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:55.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:27:55.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:27:55.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:27:55.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:27:55.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:55.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:55.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:55.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:55.932 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:27:55.932 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:55.932 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.29007321400002, 2020-11-16T22:27:56.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:27:56.134 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.134 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:27:56.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:27:56.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:27:56.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:27:56.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:27:56.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28988551200001, 2020-11-16T22:27:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:27:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:27:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28969355800001, 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:27:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:27:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:27:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28953099400002, 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:27:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:27:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:27:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28935513200003, 2020-11-16T22:27:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:27:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:27:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:27:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:27:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28918708700002, 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:27:56.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:27:56.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:27:56.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 278.28894595700001, 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:56.791 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:27:56.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:27:56.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:27:56.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.798 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:56.799 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:56.799 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:27:56.801 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:56.814+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:56.801 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:56.901 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:56.915+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:56.931 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:27:56.938 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:56.938 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:56.939 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:27:56.939 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:56.948 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:57.002 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.016+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.072 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:27:57.072 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:27:57.072 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:27:57.073 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:27:57.073 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:27:57.081 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:57.086 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:57.103 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.117+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.107 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:57.107 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:27:57.107 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:27:57.108 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:57.108 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:57.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:57.204 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.218+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.205 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:57.205 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:57.214 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:57.305 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.319+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.350 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:57.351 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:57.360 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:57.442 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.457+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.507 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:57.521+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:57.524 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:57.524 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:57.533 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:57.608 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:57.622+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:57.698 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:57.698 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:57.707 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:57.708 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.723+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:57.810 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:57.824+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:57.834 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:57.834 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:57.843 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:57.911 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:57.925+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:58.000 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [2] live_osds: [0, 1] 2020-11-16T22:27:58.000 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2020-11-16T22:27:58.000 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.2 doing powercycle of ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:27:58.001 INFO:teuthology.orchestra.console:Power on smithi122 2020-11-16T22:27:58.001 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:27:58.011 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.011 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.013 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.027+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.017 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:58.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:27:58.115 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.129+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.182 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.182 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.191 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:58.232 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.246+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.316 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.330+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.318 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.318 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.327 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:58.418 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.432+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.450 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.450 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.459 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:58.519 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:58.533+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:58.584 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.584 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.593 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:27:58.634 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:58.648+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:58.720 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:58.733+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:58.720 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.720 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.729 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:27:58.802 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:27:58.821 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:58.835+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:58.894 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:58.894 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:58.903 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:27:58.921 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:58.935+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:59.022 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:59.036+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:59.028 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:59.028 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:59.037 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:27:59.111 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:27:59.122 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:59.136+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:59.164 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:27:59.164 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:27:59.173 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:27:59.222 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:59.237+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:59.250 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:27:59.270 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:27:59.270 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:27:59.270 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:27:59.271 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:27:59.271 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:27:59.272 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:27:59.273 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:27:59.299 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:59.299 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:27:59.299 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:27:59.299 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:27:59.299 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:27:59.308 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:27:59.349 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:27:59.363+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:27:59.424 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:27:59.438+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:27:59.474 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:27:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:27:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:27:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:27:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.39840201800001, 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:27:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:27:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:27:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.28697233700001, 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:27:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:27:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:27:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97853578600001, 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:27:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:27:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:27:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:27:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:27:59.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:59.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:27:59.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:27:59.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:27:59.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:27:59.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:27:59.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:27:59.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:27:59.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:27:59.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:27:59.906 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:27:59.907 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:27:59.907 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.978420486, 2020-11-16T22:28:00.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:28:00.110 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:00.110 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:00.110 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:00.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:00.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:00.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:00.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:28:00.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:00.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:28:00.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:28:00.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:00.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:28:00.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:28:00.322 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:00.323 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:00.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:00.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.526 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:00.526 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:00.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97831172600002, 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:00.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:28:00.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:00.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:28:00.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:00.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:28:00.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:00.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:28:00.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:28:00.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:00.738 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:00.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97822937699999, 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:28:00.739 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:00.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:28:00.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:00.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:28:00.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:28:00.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:00.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:28:00.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:00.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:28:00.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:00.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:00.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:00.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:28:00.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:28:01.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:01.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:01.155 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.978141569, 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:28:01.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:01.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:28:01.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:28:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:28:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:28:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:28:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97804703000003, 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:28:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:01.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:28:01.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:28:01.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:28:01.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:28:01.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:28:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97796084399999, 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:28:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:01.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:28:01.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:28:01.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:28:01.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:28:01.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.977860459, 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:01.998 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:01.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:02.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:28:02.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:28:02.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:28:02.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:02.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:28:02.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:28:02.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:02.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97274713799999, 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671, 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.414 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514463+0000", 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2963000000000002e-05 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514506+0000", 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5280000000000003e-06 2020-11-16T22:28:02.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514511+0000", 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1880000000000002e-06 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514519+0000", 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7520000000000004e-06 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514524+0000", 2020-11-16T22:28:02.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1096000000000001e-05 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514565+0000", 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8904999999999999e-05 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514584+0000", 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001244766 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515829+0000", 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3473000000000006e-05 2020-11-16T22:28:02.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515902+0000", 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001439671 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97264096499998, 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.422 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:28:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:28:02.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:28:02.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:28:02.631 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97246695500002, 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:28:02.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:28:02.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:2 3.7 e16/14 3:f1f69726:::400.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:28:02.639 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97237067600003, 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001, 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514839+0000", 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4650999999999999e-05 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514934+0000", 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0081000000000001e-05 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514944+0000", 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4148000000000001e-05 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514958+0000", 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9730000000000002e-06 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514964+0000", 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1705e-05 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515016+0000", 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019160189999999999 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516953+0000", 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2149000000000002e-05 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517045+0000", 2020-11-16T22:28:02.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022061400000000001 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(3.2 epoch 16/14 rep_tid 8 entries 16'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:35 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2020-11-16T22:28:02.647 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97218297400002, 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401, 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515027+0000", 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4005000000000002e-05 2020-11-16T22:28:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515081+0000", 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3281e-05 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515094+0000", 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1149e-05 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515115+0000", 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9967999999999999e-05 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515135+0000", 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4321999999999998e-05 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515200+0000", 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016936760000000001 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516893+0000", 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001866401 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97199102000002, 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002, 2020-11-16T22:28:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515219+0000", 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9616999999999993e-05 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515298+0000", 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0521e-05 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515309+0000", 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3889e-05 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515323+0000", 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9039999999999997e-06 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515329+0000", 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00040774499999999999 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515737+0000", 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5534e-05 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515762+0000", 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001406654 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517169+0000", 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8978999999999999e-05 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517228+0000", 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020088430000000002 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24 3.7 e16/14 3:e0b41d06:::609.00000000:head v 16'4, mlcod=0'0)", 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97182845600003, 2020-11-16T22:28:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999, 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515381+0000", 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4316999999999999e-05 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515466+0000", 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9220000000000003e-06 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515475+0000", 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3559e-05 2020-11-16T22:28:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515488+0000", 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7350000000000004e-06 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515493+0000", 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5894000000000002e-05 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515569+0000", 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8940000000000002e-05 2020-11-16T22:28:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515588+0000", 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014616360000000001 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517049+0000", 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2508000000000005e-05 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517112+0000", 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017305109999999999 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.669 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:20 3.4 e16/14 3:3ed09add:::607.00000000:head v 16'5, mlcod=0'0)", 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97165259399998, 2020-11-16T22:28:02.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855, 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515557+0000", 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5822999999999996e-05 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515633+0000", 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5048000000000001e-05 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515648+0000", 2020-11-16T22:28:02.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2595000000000001e-05 2020-11-16T22:28:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515661+0000", 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8859999999999998e-06 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515668+0000", 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039626100000000002 2020-11-16T22:28:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516064+0000", 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7425e-05 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516081+0000", 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011498579999999999 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517231+0000", 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2959000000000001e-05 2020-11-16T22:28:02.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517264+0000", 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001706855 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.878 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:25 3.7 e16/14 3:fe07bba1:::100.00000000.inode:head v 16'5, mlcod=0'0)", 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97148454900002, 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791, 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.879 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515725+0000", 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101776 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515827+0000", 2020-11-16T22:28:02.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5319999999999995e-06 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515837+0000", 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5124e-05 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515852+0000", 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7742999999999999e-05 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515869+0000", 2020-11-16T22:28:02.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100351 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515970+0000", 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5635000000000001e-05 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515995+0000", 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016425190000000001 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517638+0000", 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2111000000000002e-05 2020-11-16T22:28:02.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517700+0000", 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001974791 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:3 3.4 e16/14 3:287034c4:::1.00000000.inode:head v 16'6, mlcod=0'0)", 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.97124341900002, 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926, 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:02.887 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515966+0000", 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9052000000000002e-05 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516035+0000", 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0380000000000005e-06 2020-11-16T22:28:02.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4902000000000001e-05 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516059+0000", 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7489999999999998e-06 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516067+0000", 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022392700000000001 2020-11-16T22:28:02.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516291+0000", 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1487999999999999e-05 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:02.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516343+0000", 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001321453 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517664+0000", 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4317000000000003e-05 2020-11-16T22:28:02.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517708+0000", 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001741926 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:02.895 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:02.895 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:02.895 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:02.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:28:02.906 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:28:02.907 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:02.910+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:02.908 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:02.996 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.011+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.045 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:03.045 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:03.046 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:03.046 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:03.055 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:03.097 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.111+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.101 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:03.107 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2020-11-16T22:28:03.121 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:03.121 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:03.122 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:03.122 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:03.122 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:03.124 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:03.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:03.180 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.181 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.190 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:03.197 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.212+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.208 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi122 2020-11-16T22:28:03.208 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi122 2020-11-16T22:28:03.268 DEBUG:teuthology.orchestra.console:expect: smithi122 login 2020-11-16T22:28:03.299 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.313+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.314 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.314 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~43s 2020-11-16T22:28:03.323 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:03.399 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.414+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.459 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.459 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.469 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:03.510 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.524+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.600 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.615+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.634 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.635 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.644 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:03.701 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:03.715+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:03.802 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.816+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.808 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.808 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.817 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:03.903 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:03.917+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:03.945 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:03.945 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:03.954 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:04.004 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.018+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.078 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.079 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.088 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:04.129 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.143+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.205 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:04.220+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:04.254 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.254 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.263 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:04.306 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.320+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.390 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.390 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.400 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:04.407 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.421+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.509 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.523+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.523 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.523 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.532 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:04.610 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.624+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.657 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.657 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.666 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:04.711 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:04.726+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:04.792 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.792 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.801 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:04.845 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.859+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.908 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:28:04.913 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:04.926+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:04.926 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:04.926 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:04.935 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:05.018 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.032+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.099 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:05.100 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:05.108 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:05.125 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:05.150 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.164+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.215 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.229+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.236 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:05.237 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:05.245 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:05.284 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:05.305 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:05.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:05.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:05.317 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.331+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.385 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:05.386 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:05.394 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:05.439 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:05.453+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:05.518 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:05.532+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.521 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 295.44526633200002, 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:28:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 295.33383665100001, 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.530 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:28:05.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.02540010000001, 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.538 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:28:05.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:28:05.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.02528480000001, 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:28:05.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:28:05.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.02517604000002, 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:28:05.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:28:05.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:28:05.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.567 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.025093691, 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:05.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:28:05.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:28:05.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:28:05.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.02500588300001, 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:28:05.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:28:05.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:28:05.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.587 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.02491134399997, 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:05.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:28:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:28:05.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:28:05.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.597 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.024825158, 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:05.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:28:05.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:28:05.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:28:05.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:05.607 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.024724773, 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:28:05.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:28:05.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:28:05.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:18 3.4 e16/14 3:2f55791f:::606.00000000:head v 16'2, mlcod=0'0)", 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:05.617 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.01950527899999, 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998, 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514569+0000", 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1594999999999994e-05 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514650+0000", 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6689999999999995e-06 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514659+0000", 2020-11-16T22:28:05.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7988000000000001e-05 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514677+0000", 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6019999999999996e-06 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514684+0000", 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9691000000000002e-05 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514733+0000", 2020-11-16T22:28:05.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3110000000000001e-05 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514757+0000", 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022777370000000002 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517034+0000", 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4086999999999998e-05 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517098+0000", 2020-11-16T22:28:05.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025294789999999998 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:1 3.4 e16/14 3:292cf221:::200.00000000:head v 16'3, mlcod=0'0)", 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:05.625 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.01933126900002, 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999, 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514743+0000", 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5374999999999998e-05 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514788+0000", 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3290000000000001e-06 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514794+0000", 2020-11-16T22:28:05.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6649999999999992e-06 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514802+0000", 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4229999999999998e-06 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514807+0000", 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015299700000000001 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514960+0000", 2020-11-16T22:28:05.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7331999999999999e-05 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.514987+0000", 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002115418 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517102+0000", 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3810999999999997e-05 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517156+0000", 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024133499999999999 2020-11-16T22:28:05.632 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:6 4.4 4:30604137:::100000001fc.00000000:head [write 0~1492] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.800230+0000", 2020-11-16T22:28:05.633 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.7338437039999999, 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002353136, 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 6 2020-11-16T22:28:05.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800230+0000", 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800230+0000", 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0936999999999998e-05 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800271+0000", 2020-11-16T22:28:05.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4189999999999999e-06 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800278+0000", 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2917e-05 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800291+0000", 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9240000000000004e-06 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800297+0000", 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6714e-05 2020-11-16T22:28:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800343+0000", 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014559600000000001 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800489+0000", 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116889 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.800606+0000", 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00069617599999999996 2020-11-16T22:28:05.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.801302+0000", 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010801470000000001 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.802382+0000", 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4639000000000001e-05 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.802467+0000", 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116778 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.802584+0000", 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002353136 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:55 3.4 3:2e2fa760:::200.00000001:head [write 27165~53918 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.7100153360000001, 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528429999999999, 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.644 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010680599999999999 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824166+0000", 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4158000000000001e-05 2020-11-16T22:28:05.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824180+0000", 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3141e-05 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824193+0000", 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5459999999999997e-06 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824199+0000", 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7255000000000006e-05 2020-11-16T22:28:05.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824267+0000", 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8100000000000001e-05 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824345+0000", 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012491999999999999 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824470+0000", 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014246459999999999 2020-11-16T22:28:05.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.825894+0000", 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00086393699999999995 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826758+0000", 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3545000000000001e-05 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826832+0000", 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9789000000000005e-05 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826912+0000", 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528429999999999 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:56 3.4 3:2e2fa760:::200.00000001:head [write 81083~9050 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.703997942, 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047997739999999997, 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.654 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6360999999999994e-05 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830173+0000", 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5549999999999999e-06 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830179+0000", 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1950000000000001e-05 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830191+0000", 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.19e-06 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830197+0000", 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5439999999999998e-05 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830253+0000", 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103256 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830356+0000", 2020-11-16T22:28:05.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00019858699999999999 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830555+0000", 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00094351299999999999 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.831498+0000", 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032175569999999998 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834716+0000", 2020-11-16T22:28:05.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0638999999999999e-05 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834786+0000", 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9726000000000001e-05 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834876+0000", 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047997739999999997 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:57 3.4 3:2e2fa760:::200.00000001:head [write 90133~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.6961833070000001, 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045461870000000001, 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.666 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:05.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7341000000000005e-05 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837958+0000", 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1550000000000004e-06 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837964+0000", 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0485999999999999e-05 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837975+0000", 2020-11-16T22:28:05.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.164e-06 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837980+0000", 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0513999999999997e-05 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838041+0000", 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9005999999999996e-05 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838110+0000", 2020-11-16T22:28:05.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013133900000000001 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838241+0000", 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00082306499999999999 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.839064+0000", 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032202770000000001 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842284+0000", 2020-11-16T22:28:05.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3054999999999999e-05 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842347+0000", 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9784999999999995e-05 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842437+0000", 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045461870000000001 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:58 3.4 3:2e2fa760:::200.00000001:head [write 98687~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.6887875170000002, 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030695420000000002, 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.676 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1511000000000002e-05 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845338+0000", 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4130000000000003e-06 2020-11-16T22:28:05.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845345+0000", 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0242e-05 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845355+0000", 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0764999999999999e-05 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845366+0000", 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1292000000000003e-05 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845417+0000", 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7689000000000006e-05 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845515+0000", 2020-11-16T22:28:05.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00019681399999999999 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845711+0000", 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00073376599999999995 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.846445+0000", 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001779 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848224+0000", 2020-11-16T22:28:05.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2024000000000002e-05 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848276+0000", 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0025999999999996e-05 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848356+0000", 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030695420000000002 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:60 3.4 3:2e2fa760:::200.00000001:head [write 134544~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.6614546649999999, 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026483879999999998, 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4002999999999998e-05 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872674+0000", 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7479999999999999e-06 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872679+0000", 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0984e-05 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872690+0000", 2020-11-16T22:28:05.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.814e-06 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872695+0000", 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9835e-05 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872745+0000", 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102166 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872847+0000", 2020-11-16T22:28:05.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112513 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872960+0000", 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00076257600000000005 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.873722+0000", 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014148819999999999 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875137+0000", 2020-11-16T22:28:05.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2914000000000002e-05 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875190+0000", 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7953000000000004e-05 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875268+0000", 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026483879999999998 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.695 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:61 3.4 3:2e2fa760:::200.00000001:head [write 143098~1933 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:02.907614+0000", 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 2.6264603439999998, 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023727779999999999, 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907614+0000", 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907614+0000", 2020-11-16T22:28:05.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2889000000000002e-05 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907657+0000", 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6270000000000006e-06 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907665+0000", 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3139000000000001e-05 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907678+0000", 2020-11-16T22:28:05.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0779999999999998e-06 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907686+0000", 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7392999999999998e-05 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907743+0000", 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2958000000000001e-05 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907826+0000", 2020-11-16T22:28:05.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116424 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.907942+0000", 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00086024199999999999 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.908803+0000", 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010340849999999999 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.909837+0000", 2020-11-16T22:28:05.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6206999999999999e-05 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.909893+0000", 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3735999999999998e-05 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:02.909987+0000", 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023727779999999999 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.705 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.21349770900000001, 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:05.706 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:05.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:05.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:05.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:05.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:05.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:05.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:05.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:05.715 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:05.717 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.730+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.717 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:05.818 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:05.832+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:05.846 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:05.847 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:05.855 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:05.919 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:05.933+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:05.981 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:05.981 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:05.981 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:28:05.981 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:28:05.981 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:05.990 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:06.017 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:28:06.019 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:06.034+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:06.120 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:06.134+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:06.157 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:06.157 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:06.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:4 3.7 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66200165399999, 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996, 2020-11-16T22:28:06.158 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:06.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508121+0000", 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2243999999999997e-05 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:06.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508164+0000", 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508170+0000", 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0883999999999999e-05 2020-11-16T22:28:06.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508181+0000", 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5447e-05 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:06.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508197+0000", 2020-11-16T22:28:06.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0930000000000002e-05 2020-11-16T22:28:06.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508248+0000", 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508254+0000", 2020-11-16T22:28:06.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001582176 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509836+0000", 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017347200000000001 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510010+0000", 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032401700000000001 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:06.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510334+0000", 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043742570000000003 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514708+0000", 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6449000000000002e-05 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514755+0000", 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2448999999999996e-05 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514837+0000", 2020-11-16T22:28:06.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5692999999999997e-05 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514893+0000", 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067713419999999996 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.169 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:17 3.4 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66188944700002, 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999, 2020-11-16T22:28:06.170 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:06.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508234+0000", 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7926000000000002e-05 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:06.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508272+0000", 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3300000000000001e-06 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508279+0000", 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0011e-05 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508289+0000", 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6030000000000001e-06 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:06.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508292+0000", 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8467000000000003e-05 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508331+0000", 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7209999999999999e-06 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508337+0000", 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001358354 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509695+0000", 2020-11-16T22:28:06.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175525 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509871+0000", 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024005599999999999 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510111+0000", 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0047782160000000001 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514889+0000", 2020-11-16T22:28:06.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2455000000000001e-05 2020-11-16T22:28:06.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:06.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514911+0000", 2020-11-16T22:28:06.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4654000000000006e-05 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514996+0000", 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0188e-05 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515056+0000", 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068225059999999999 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66179920899998, 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:06.385 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:06.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:06.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:06.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:06.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:06.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:06.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:06.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:06.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:06.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:06.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:06.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:06.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:06.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:06.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66168256600002, 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:06.803 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:06.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:06.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:06.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:06.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:06.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:06.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:06.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:07.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:07.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:07.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:07.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:07.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:07.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66153419699998, 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:07.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:07.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:07.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:07.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:07.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:07.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:07.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:07.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:07.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:07.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:07.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21 3.6 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:28:07.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66134482799998, 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001, 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:07.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508778+0000", 2020-11-16T22:28:07.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010302999999999999 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508881+0000", 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6179999999999995e-06 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508890+0000", 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6294999999999999e-05 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508906+0000", 2020-11-16T22:28:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7749999999999998e-06 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508912+0000", 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0948999999999999e-05 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509003+0000", 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5889999999999997e-06 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509010+0000", 2020-11-16T22:28:07.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00076899499999999997 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509779+0000", 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019657400000000001 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509976+0000", 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000244097 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510220+0000", 2020-11-16T22:28:07.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025075929999999998 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512728+0000", 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022415400000000002 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514969+0000", 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7627999999999997e-05 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515037+0000", 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9556999999999999e-05 2020-11-16T22:28:07.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:07.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515096+0000", 2020-11-16T22:28:07.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063182400000000001 2020-11-16T22:28:07.647 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.850 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:07.850 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:07.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.850 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66113090599998, 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:07.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:28:07.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:28:08.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:28:08.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:28:08.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:28:08.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:08.065 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66093200500001, 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:08.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:28:08.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:28:08.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:28:08.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:28:08.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:08.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66075076800001, 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:08.481 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:08.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:08.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:08.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:08.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:08.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:08.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.66012345399997, 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:08.694 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:08.695 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:08.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:08.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:08.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:08.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:28:08.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:28:08.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:28:08.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:28:08.906 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65982611099997, 2020-11-16T22:28:08.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:28:09.110 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:09.110 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:09.110 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:09.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:09.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:28:09.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:09.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:28:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:28:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:28:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:28:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:26 3.3 e16/14 3:c91d4a1d:::mds0_inotable:head v 16'2, mlcod=0'0)", 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65941929899998, 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999, 2020-11-16T22:28:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510704+0000", 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9781e-05 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510794+0000", 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2782000000000001e-05 2020-11-16T22:28:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510815+0000", 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1133e-05 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510827+0000", 2020-11-16T22:28:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016746299999999999 2020-11-16T22:28:09.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510994+0000", 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6804e-05 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511021+0000", 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040137219999999999 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515035+0000", 2020-11-16T22:28:09.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2464000000000004e-05 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515097+0000", 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043933169999999999 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.533 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65932008099998, 2020-11-16T22:28:09.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:09.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:09.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:09.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:28:09.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:09.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:28:09.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:28:09.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:28:09.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:09.746 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65912511499999, 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:09.747 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:09.748 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:09.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:09.950 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:09.950 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:09.952 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:09.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:28:09.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:28:09.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:09.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:10.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:28:10.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:28:10.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65856404300001, 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:10.163 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:10.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:28:10.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:28:10.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:10.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:28:10.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:28:10.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:10.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:34 3.7 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:10.372 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65840556199998, 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999, 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511717+0000", 2020-11-16T22:28:10.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1780999999999998e-05 2020-11-16T22:28:10.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:10.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511769+0000", 2020-11-16T22:28:10.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5320000000000001e-06 2020-11-16T22:28:10.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511777+0000", 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.362e-05 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511790+0000", 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7699999999999998e-06 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511796+0000", 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031388240000000001 2020-11-16T22:28:10.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514935+0000", 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018309899999999999 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515118+0000", 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041480909999999996 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519266+0000", 2020-11-16T22:28:10.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0075487169999999999 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:35 3.2 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:10.583 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.658272651, 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997, 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:10.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511850+0000", 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2877000000000002e-05 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511903+0000", 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4540000000000001e-06 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:10.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511911+0000", 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0928999999999999e-05 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511932+0000", 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8950000000000003e-06 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:10.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511938+0000", 2020-11-16T22:28:10.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045371499999999999 2020-11-16T22:28:10.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512391+0000", 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025627199999999999 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512648+0000", 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057394009999999999 2020-11-16T22:28:10.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518387+0000", 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065365429999999997 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:10.792 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:36 3.6 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 288.65810404199999, 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998, 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:10.793 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:10.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512019+0000", 2020-11-16T22:28:10.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3044999999999998e-05 2020-11-16T22:28:10.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512072+0000", 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4469999999999995e-06 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512081+0000", 2020-11-16T22:28:10.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3121e-05 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512095+0000", 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512101+0000", 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066987299999999997 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512771+0000", 2020-11-16T22:28:11.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029465399999999999 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513066+0000", 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044664029999999999 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517532+0000", 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055129139999999998 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:40 3.4 3:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e19)", 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 269.422273254, 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002, 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:11.004 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747850+0000", 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6168000000000003e-05 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747886+0000", 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7950000000000001e-06 2020-11-16T22:28:11.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747890+0000", 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4390000000000006e-06 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:11.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747899+0000", 2020-11-16T22:28:11.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7480000000000002e-06 2020-11-16T22:28:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747902+0000", 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3952000000000001e-05 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:11.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747956+0000", 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7330000000000003e-06 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.747961+0000", 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023195680000000002 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750280+0000", 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3100000000000001e-05 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750353+0000", 2020-11-16T22:28:11.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9628000000000002e-05 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750453+0000", 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046040099999999999 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.750913+0000", 2020-11-16T22:28:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071694300000000005 2020-11-16T22:28:11.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:11.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751630+0000", 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1374e-05 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751682+0000", 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8221e-05 2020-11-16T22:28:11.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:36.751710+0000", 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038600700000000002 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:16 4.7 4:e15a914f:::10000000207.00000000:head [write 0~240] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:01.813593+0000", 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4.3565300059999998, 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040564260000000001, 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:11.421 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 16 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813593+0000", 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813593+0000", 2020-11-16T22:28:11.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1547999999999997e-05 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813625+0000", 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3269999999999997e-06 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813631+0000", 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2013e-05 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813643+0000", 2020-11-16T22:28:11.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7449999999999998e-06 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813650+0000", 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4160999999999996e-05 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813734+0000", 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000135494 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.813869+0000", 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014621600000000001 2020-11-16T22:28:11.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.814016+0000", 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015872029999999999 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.815603+0000", 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001765693 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.817368+0000", 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000152647 2020-11-16T22:28:11.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.817521+0000", 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000128379 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:11.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:01.817649+0000", 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040564260000000001 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:11.431 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:11.432 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:11.433 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:28:11.434 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:11.447+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:11.435 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:11.534 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:11.548+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:11.572 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:11.573 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:11.573 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:28:11.573 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:11.582 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:11.611 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:11.631 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:11.631 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:11.632 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:11.632 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:11.632 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:11.634 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:11.635 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:11.648+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:11.734 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:11.749+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:11.746 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:11.746 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:11.755 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:11.836 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:11.850+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:11.879 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:11.879 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:11.888 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:11.936 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:11.951+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.025 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.026 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.032 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:28:12.035 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:12.037 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:12.051+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.138 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:12.153+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.160 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.160 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.169 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:12.239 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:12.253+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2020-11-16T22:28:12.292 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.292 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.301 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:12.340 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:12.354+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:12.428 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.428 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.437 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:12.480 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:12.494+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.541 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:12.555+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:12.587 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.588 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.601 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:12.642 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:12.656+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:12.742 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:12.756+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:12.786 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.787 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.798 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:12.843 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:12.857+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:12.939 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:12.939 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:12.944 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:12.958+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:12.949 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:13.045 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:13.059+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:13.125 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:13.125 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:13.135 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:13.178 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:13.192+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:13.247 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:13.261+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:13.283 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:13.283 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:13.294 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:13.348 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:13.362+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:13.445 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:13.446 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:13.449 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:13.463+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:13.457 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:13.550 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:13.564+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:13.635 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:13.651 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:13.665+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:13.655 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:13.656 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:13.666 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:13.752 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:13.766+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:13.793 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:13.814 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:13.815 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:13.815 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:13.815 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:13.815 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:13.817 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:2 1.0 e9/8 1:8b2356c0:::INTEL_SSDPEDMD400G4_PHFT6204019S400BGN:head v 9'1, mlcod=0'0)", 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:13.818 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 303.741330877, 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998, 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088808+0000", 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6421999999999997e-05 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088854+0000", 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7969999999999997e-06 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088860+0000", 2020-11-16T22:28:13.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0165e-05 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088870+0000", 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088878+0000", 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5698999999999998e-05 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088914+0000", 2020-11-16T22:28:13.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.885e-05 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.088943+0000", 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0070119120000000004 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.095954+0000", 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1897999999999996e-05 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.096016+0000", 2020-11-16T22:28:13.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072084929999999998 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4166.0:4 1.0 e9/8 1:1b5ff733:::INTEL_SSDPEDMD400G4_CVFT6240002A400BGN:head v 9'2, mlcod=0'0)", 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:13.826 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 303.62990119599999, 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999, 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:13.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200238+0000", 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9483000000000001e-05 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200277+0000", 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1381e-05 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200288+0000", 2020-11-16T22:28:13.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1551e-05 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200300+0000", 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0838e-05 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200311+0000", 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3122999999999997e-05 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200394+0000", 2020-11-16T22:28:13.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5520999999999998e-05 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.200419+0000", 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034308120000000001 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203850+0000", 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100851 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:10.203951+0000", 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037135599999999999 2020-11-16T22:28:13.833 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5 3.0 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:13.834 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32146464499999, 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996, 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:13.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508674+0000", 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.316e-05 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508717+0000", 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7759999999999992e-06 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508727+0000", 2020-11-16T22:28:13.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508738+0000", 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5060000000000001e-06 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508745+0000", 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0182000000000001e-05 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508805+0000", 2020-11-16T22:28:13.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018607500000000001 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508991+0000", 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000169889 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509161+0000", 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029267189999999999 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512087+0000", 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032332810000000002 2020-11-16T22:28:13.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515321+0000", 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4539999999999999e-05 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515415+0000", 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7694e-05 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515473+0000", 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067989189999999996 2020-11-16T22:28:13.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:6 3.0 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32134934499999, 2020-11-16T22:28:13.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996, 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:13.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508789+0000", 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0394e-05 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508830+0000", 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508837+0000", 2020-11-16T22:28:13.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3190000000000001e-05 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508850+0000", 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7010000000000001e-06 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508856+0000", 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00067719199999999996 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509533+0000", 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4485000000000003e-05 2020-11-16T22:28:13.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509608+0000", 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010919799999999999 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509717+0000", 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0023762000000000002 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512093+0000", 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034353040000000001 2020-11-16T22:28:13.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515528+0000", 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7302e-05 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515606+0000", 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9858999999999997e-05 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515666+0000", 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0068762679999999996 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.853 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7 3.0 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.321240585, 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003, 2020-11-16T22:28:13.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508898+0000", 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7734999999999997e-05 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508936+0000", 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7659999999999998e-06 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508941+0000", 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5250000000000005e-06 2020-11-16T22:28:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508949+0000", 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1139999999999999e-06 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508953+0000", 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000997747 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509951+0000", 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000142478 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510093+0000", 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158276 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510252+0000", 2020-11-16T22:28:13.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018463590000000001 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512098+0000", 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003626046 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515724+0000", 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7420000000000001e-05 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515802+0000", 2020-11-16T22:28:13.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8014999999999999e-05 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.515860+0000", 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0069614810000000003 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8 3.0 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32115823599997, 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997, 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.864 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.508980+0000", 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0726999999999998e-05 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509021+0000", 2020-11-16T22:28:13.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4170000000000002e-06 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509026+0000", 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509033+0000", 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.988e-06 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509036+0000", 2020-11-16T22:28:13.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001532406 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510569+0000", 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0235999999999996e-05 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510639+0000", 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013571300000000001 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.510775+0000", 2020-11-16T22:28:13.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021194209999999998 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512894+0000", 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003387223 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516281+0000", 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.687e-05 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516368+0000", 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9438999999999998e-05 2020-11-16T22:28:13.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.516427+0000", 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0074470229999999997 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9 3.0 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32107042799998, 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992, 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.874 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509068+0000", 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7784000000000002e-05 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509116+0000", 2020-11-16T22:28:13.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0030000000000002e-06 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509121+0000", 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3579999999999992e-06 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509130+0000", 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8199999999999996e-06 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509135+0000", 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001969246 2020-11-16T22:28:13.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511104+0000", 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017295099999999999 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511277+0000", 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015645700000000001 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511434+0000", 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014654449999999999 2020-11-16T22:28:13.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512899+0000", 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048737679999999997 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517773+0000", 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3923000000000004e-05 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517847+0000", 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3272000000000001e-05 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517900+0000", 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0088320269999999992 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:10 3.0 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32097588900001, 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006, 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.884 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509163+0000", 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0895000000000001e-05 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509204+0000", 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0330000000000002e-06 2020-11-16T22:28:13.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509208+0000", 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6289999999999999e-06 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509216+0000", 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.033e-06 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509219+0000", 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024943669999999999 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511714+0000", 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5141000000000005e-05 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511779+0000", 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010051 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.511879+0000", 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010237829999999999 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512903+0000", 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050496229999999996 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.517953+0000", 2020-11-16T22:28:13.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0300000000000002e-05 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518013+0000", 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7021000000000002e-05 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518070+0000", 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0089073350000000006 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11 3.0 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32088970299998, 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004, 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509249+0000", 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3803000000000001e-05 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6489999999999999e-06 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509297+0000", 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7029999999999995e-06 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509306+0000", 2020-11-16T22:28:13.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.46e-06 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509313+0000", 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002818269 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512131+0000", 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000124246 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512255+0000", 2020-11-16T22:28:13.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8843000000000002e-05 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512354+0000", 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00055339600000000005 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512907+0000", 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052182579999999999 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518126+0000", 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5038999999999998e-05 2020-11-16T22:28:13.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518201+0000", 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7827e-05 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518258+0000", 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090094930000000004 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:12 3.0 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 296.32078931799998, 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001, 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509349+0000", 2020-11-16T22:28:13.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5825999999999998e-05 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509395+0000", 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2950000000000004e-06 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509400+0000", 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2777e-05 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509412+0000", 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4649999999999996e-06 2020-11-16T22:28:13.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.509417+0000", 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031462149999999999 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512563+0000", 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512616+0000", 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7606e-05 2020-11-16T22:28:13.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.512714+0000", 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043233099999999999 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.513146+0000", 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051612020000000002 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518307+0000", 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8101000000000002e-05 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518396+0000", 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5295000000000004e-05 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:23:17.518461+0000", 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0091114730000000001 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:55 3.4 3:2e2fa760:::200.00000001:head [write 27165~53918 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:13.913 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 12.006079881, 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528429999999999, 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824059+0000", 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010680599999999999 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824166+0000", 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4158000000000001e-05 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824180+0000", 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3141e-05 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824193+0000", 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5459999999999997e-06 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824199+0000", 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7255000000000006e-05 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824267+0000", 2020-11-16T22:28:13.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8100000000000001e-05 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824345+0000", 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012491999999999999 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.824470+0000", 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014246459999999999 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.825894+0000", 2020-11-16T22:28:13.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00086393699999999995 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826758+0000", 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3545000000000001e-05 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826832+0000", 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9789000000000005e-05 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.826912+0000", 2020-11-16T22:28:13.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528429999999999 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:56 3.4 3:2e2fa760:::200.00000001:head [write 81083~9050 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:13.923 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 12.000062486999999, 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047997739999999997, 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:13.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830076+0000", 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6360999999999994e-05 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830173+0000", 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5549999999999999e-06 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830179+0000", 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1950000000000001e-05 2020-11-16T22:28:13.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830191+0000", 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.19e-06 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830197+0000", 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5439999999999998e-05 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830253+0000", 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103256 2020-11-16T22:28:13.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830356+0000", 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00019858699999999999 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.830555+0000", 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00094351299999999999 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.831498+0000", 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032175569999999998 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834716+0000", 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0638999999999999e-05 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834786+0000", 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9726000000000001e-05 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.834876+0000", 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047997739999999997 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.932 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:57 3.4 3:2e2fa760:::200.00000001:head [write 90133~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.992247852, 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045461870000000001, 2020-11-16T22:28:13.933 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837891+0000", 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7341000000000005e-05 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837958+0000", 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1550000000000004e-06 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837964+0000", 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0485999999999999e-05 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837975+0000", 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.164e-06 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.837980+0000", 2020-11-16T22:28:13.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0513999999999997e-05 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838041+0000", 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9005999999999996e-05 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838110+0000", 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013133900000000001 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.838241+0000", 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00082306499999999999 2020-11-16T22:28:13.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.839064+0000", 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032202770000000001 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842284+0000", 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3054999999999999e-05 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842347+0000", 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9784999999999995e-05 2020-11-16T22:28:13.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.842437+0000", 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045461870000000001 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:58 3.4 3:2e2fa760:::200.00000001:head [write 98687~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.984852062, 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030695420000000002, 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845287+0000", 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1511000000000002e-05 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845338+0000", 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4130000000000003e-06 2020-11-16T22:28:13.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845345+0000", 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0242e-05 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845355+0000", 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0764999999999999e-05 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845366+0000", 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1292000000000003e-05 2020-11-16T22:28:13.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845417+0000", 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.7689000000000006e-05 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845515+0000", 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00019681399999999999 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.845711+0000", 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00073376599999999995 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.846445+0000", 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001779 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848224+0000", 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2024000000000002e-05 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848276+0000", 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0025999999999996e-05 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.848356+0000", 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030695420000000002 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:60 3.4 3:2e2fa760:::200.00000001:head [write 134544~8554 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e85)", 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.957519209999999, 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026483879999999998, 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872619+0000", 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4002999999999998e-05 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872674+0000", 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7479999999999999e-06 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872679+0000", 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0984e-05 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872690+0000", 2020-11-16T22:28:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.814e-06 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872695+0000", 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9835e-05 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872745+0000", 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102166 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872847+0000", 2020-11-16T22:28:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112513 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.872960+0000", 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00076257600000000005 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.873722+0000", 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014148819999999999 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875137+0000", 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2914000000000002e-05 2020-11-16T22:28:13.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875190+0000", 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7953000000000004e-05 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:01.875268+0000", 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026483879999999998 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.962 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 8.5095622540000004, 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:13.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:13.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:13.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:87 3.4 3:2e2fa760:::200.00000001:head [write 168557~31520 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e86)", 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:09.206190+0000", 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6239486080000001, 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028998000000000001, 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.972 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206190+0000", 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206190+0000", 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014116700000000001 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206331+0000", 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9090000000000003e-06 2020-11-16T22:28:13.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206338+0000", 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3888e-05 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206352+0000", 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8940000000000005e-06 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206362+0000", 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7544000000000001e-05 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206450+0000", 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000146069 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206596+0000", 2020-11-16T22:28:13.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000208817 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206804+0000", 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001324892 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208129+0000", 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00076115600000000003 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208890+0000", 2020-11-16T22:28:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5068999999999996e-05 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208966+0000", 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012439499999999999 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209090+0000", 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028998000000000001 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:89 3.4 3:2e2fa760:::200.00000001:head [write 200077~1601 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e86)", 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:09.206501+0000", 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6236381939999998, 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029345780000000002, 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206501+0000", 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206501+0000", 2020-11-16T22:28:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8414999999999997e-05 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206539+0000", 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1429999999999997e-06 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206544+0000", 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0099999999999998e-06 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206551+0000", 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.529e-06 2020-11-16T22:28:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206555+0000", 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010055159999999999 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.207560+0000", 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2479999999999997e-05 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.207633+0000", 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012762799999999999 2020-11-16T22:28:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.207760+0000", 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0013567130000000001 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209117+0000", 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112438 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209229+0000", 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1126999999999998e-05 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209311+0000", 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012457899999999999 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209435+0000", 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029345780000000002 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:90 3.4 3:2e2fa760:::200.00000001:head [write 201678~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e86)", 2020-11-16T22:28:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:09.206587+0000", 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6235513199999998, 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032891489999999999, 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206587+0000", 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206587+0000", 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5596999999999997e-05 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206633+0000", 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7060000000000003e-06 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206638+0000", 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.559e-06 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206645+0000", 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8509999999999999e-06 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.206648+0000", 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015163100000000001 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208164+0000", 2020-11-16T22:28:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000125468 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208290+0000", 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000184131 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.208474+0000", 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00066547000000000004 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209139+0000", 2020-11-16T22:28:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00052095500000000003 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209660+0000", 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5223999999999997e-05 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209736+0000", 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014087799999999999 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:09.209877+0000", 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032891489999999999 2020-11-16T22:28:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.5090593659999998, 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:14.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:14.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:14.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:14.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:14.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.010 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:14.012 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:14.012 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:14.013 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:14.025+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:14.013 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:14.043 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:28:14.111 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:14.126+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:14.152 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:14.152 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:14.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.152 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.4471.0:2142 4.0 e87/82 4:1a854a82:::10000000a89.00000000:head v 87'267, mlcod=87'265)", 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.165925+0000", 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0010210110000000001, 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010368389999999999, 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165925+0000", 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165925+0000", 2020-11-16T22:28:14.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3793e-05 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165959+0000", 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3929999999999999e-06 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165964+0000", 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1460000000000002e-06 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165973+0000", 2020-11-16T22:28:14.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0270000000000003e-06 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.165978+0000", 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6024e-05 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.166004+0000", 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.613e-05 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.166020+0000", 2020-11-16T22:28:14.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5513000000000005e-05 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2020-11-16T22:28:14.159 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:14.162 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:14.212 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:14.226+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:14.303 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:14.303 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:28:14.304 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:28:14.304 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:14.313 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:14.327+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:14.314 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:14.413 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:14.427+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:14.502 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.503 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00724844899997, 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:14.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:14.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:14.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:14.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:14.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00713180600002, 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.515 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:14.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:14.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:14.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:14.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.525 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00698343699997, 2020-11-16T22:28:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:14.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:14.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:14.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:22 3.6 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.537 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00658014599998, 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006, 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508992+0000", 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5593000000000004e-05 2020-11-16T22:28:14.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509088+0000", 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0948e-05 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509109+0000", 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8139999999999999e-05 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509127+0000", 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6609999999999995e-06 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509136+0000", 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2357000000000001e-05 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509179+0000", 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509186+0000", 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001771316 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510957+0000", 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119358 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511077+0000", 2020-11-16T22:28:14.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098860399999999992 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512065+0000", 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030486020000000001 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515114+0000", 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0019330689999999999 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517047+0000", 2020-11-16T22:28:14.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6452999999999997e-05 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517093+0000", 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7403e-05 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.517141+0000", 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081486430000000006 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:23 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.006381245, 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007, 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509191+0000", 2020-11-16T22:28:14.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7087000000000002e-05 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509268+0000", 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509277+0000", 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5926999999999998e-05 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509293+0000", 2020-11-16T22:28:14.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509300+0000", 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017144459999999999 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511014+0000", 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000201516 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511216+0000", 2020-11-16T22:28:14.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000484351 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511700+0000", 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030160339999999999 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514716+0000", 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035528019999999999 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518269+0000", 2020-11-16T22:28:14.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6812999999999998e-05 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518346+0000", 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4042999999999997e-05 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518410+0000", 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092188400000000007 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00620000800001, 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:14.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:14.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:14.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:14.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:1 3.4 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.568 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00557269400002, 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005, 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510000+0000", 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014768 2020-11-16T22:28:14.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510147+0000", 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1784999999999999e-05 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510169+0000", 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2563000000000001e-05 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510192+0000", 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3355e-05 2020-11-16T22:28:14.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510205+0000", 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001785848 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511991+0000", 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015333400000000001 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512144+0000", 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030725700000000002 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512451+0000", 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034495569999999998 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515901+0000", 2020-11-16T22:28:14.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028050140000000002 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518706+0000", 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2797000000000003e-05 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518759+0000", 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5672e-05 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518794+0000", 2020-11-16T22:28:14.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087948620000000005 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:2 3.7 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.578 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00527535100002, 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995, 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510297+0000", 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040252300000000001 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510699+0000", 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7590000000000003e-06 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510708+0000", 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2472000000000001e-05 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510731+0000", 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7680000000000008e-06 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510738+0000", 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001287177 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512026+0000", 2020-11-16T22:28:14.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020612899999999999 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512232+0000", 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048791200000000001 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512720+0000", 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028948340000000002 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515615+0000", 2020-11-16T22:28:14.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028487819999999998 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518463+0000", 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6379999999999999e-05 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518520+0000", 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0511999999999999e-05 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518550+0000", 2020-11-16T22:28:14.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082532479999999995 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:3 3.4 3:287034c4:::1.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:14.588 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00476932100003, 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995, 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:14.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510803+0000", 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010173100000000001 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510905+0000", 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8249999999999994e-06 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510914+0000", 2020-11-16T22:28:14.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1058999999999999e-05 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510935+0000", 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2400000000000001e-06 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510942+0000", 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0033767319999999999 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514319+0000", 2020-11-16T22:28:14.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112698 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514431+0000", 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102396 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514534+0000", 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015519279999999999 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516086+0000", 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030499070000000001 2020-11-16T22:28:14.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519135+0000", 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1780999999999995e-05 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519217+0000", 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2021999999999997e-05 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519279+0000", 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084763189999999995 2020-11-16T22:28:14.597 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:25 3.7 3:fe07bba1:::100.00000000.inode:head [writefull 0~553] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00457435499999, 2020-11-16T22:28:14.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003, 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.510998+0000", 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4867e-05 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.600 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511073+0000", 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5179999999999993e-06 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511082+0000", 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6121999999999999e-05 2020-11-16T22:28:14.601 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511098+0000", 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7989999999999996e-06 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.602 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511105+0000", 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003043081 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514148+0000", 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014743499999999999 2020-11-16T22:28:14.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514296+0000", 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011762200000000001 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514413+0000", 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016482999999999999 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516062+0000", 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028938660000000001 2020-11-16T22:28:14.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518956+0000", 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5023000000000003e-05 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519031+0000", 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8659000000000003e-05 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519079+0000", 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080812920000000003 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.607 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:31 3.4 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 297.00401328300001, 2020-11-16T22:28:14.608 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998, 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511559+0000", 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8067999999999999e-05 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.610 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511627+0000", 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9449999999999999e-06 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511636+0000", 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7941999999999999e-05 2020-11-16T22:28:14.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511654+0000", 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4359999999999996e-06 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511661+0000", 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034414010000000002 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515103+0000", 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026534300000000002 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515368+0000", 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040929299999999998 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519461+0000", 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079020649999999998 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:1746 4.7 4:f9737ae1:::100000008fd.00000000:head [write 0~277] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.615 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:13.865356+0000", 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.65021611800000001, 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0088377779999999993, 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.616 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 1746 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865356+0000", 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865356+0000", 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8340999999999999e-05 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865385+0000", 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.481e-06 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865389+0000", 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1708999999999999e-05 2020-11-16T22:28:14.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865401+0000", 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0590000000000002e-06 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865406+0000", 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1168999999999999e-05 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865437+0000", 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8208000000000006e-05 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865535+0000", 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116282 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.865651+0000", 2020-11-16T22:28:14.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00059731400000000005 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.866249+0000", 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0077460740000000004 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.873995+0000", 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0153000000000004e-05 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.874065+0000", 2020-11-16T22:28:14.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012898799999999999 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.874194+0000", 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0088377779999999993 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.625 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.4471.0:1801 4.0 e87/82 4:08bfbaff:::10000000934.00000000:head v 87'226, mlcod=87'224)", 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:13.905298+0000", 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.61027394899999998, 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0078804219999999998, 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.626 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905298+0000", 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905298+0000", 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5800000000000003e-05 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905334+0000", 2020-11-16T22:28:14.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3459999999999999e-06 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905340+0000", 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0223e-05 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.629 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905351+0000", 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0119999999999996e-06 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905356+0000", 2020-11-16T22:28:14.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5726999999999999e-05 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905391+0000", 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5097e-05 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.905417+0000", 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0077164870000000002 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.913133+0000", 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5729999999999998e-05 2020-11-16T22:28:14.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.913179+0000", 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0078804219999999998 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.633 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.4471.0:1805 4.4 e87/82 4:38c6ebbe:::10000000938.00000000:head v 87'188, mlcod=87'186)", 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:13.907945+0000", 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.60762685999999999, 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080878180000000001, 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.634 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.907945+0000", 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.907945+0000", 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8546e-05 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.907984+0000", 2020-11-16T22:28:14.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4119999999999998e-06 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.907990+0000", 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5079999999999994e-06 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.908000+0000", 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1499999999999998e-06 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.908005+0000", 2020-11-16T22:28:14.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8501e-05 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.908034+0000", 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4995e-05 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.908049+0000", 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0079393099999999998 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.915988+0000", 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5395999999999997e-05 2020-11-16T22:28:14.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:13.916033+0000", 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080878180000000001 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.45016358000000001, 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.642 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:14.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:14.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:14.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2030 4.7 4:ea5948e4:::10000000a19.00000000:head [write 0~32] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.652 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.086799+0000", 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.428773444, 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087360419999999994, 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.653 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2030 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086799+0000", 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086799+0000", 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3631999999999997e-05 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086852+0000", 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6339999999999996e-06 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086862+0000", 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5923e-05 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086878+0000", 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4009999999999996e-06 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086883+0000", 2020-11-16T22:28:14.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7561999999999997e-05 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.086921+0000", 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9071000000000001e-05 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.087020+0000", 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000123089 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.087143+0000", 2020-11-16T22:28:14.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00059187400000000005 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.087735+0000", 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0076848849999999998 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.095420+0000", 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6415999999999997e-05 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.095466+0000", 2020-11-16T22:28:14.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8554999999999997e-05 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.095535+0000", 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0087360419999999994 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.662 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.4471.0:2091 4.6 e87/82 4:765512e2:::10000000a56.00000000:head v 87'249, mlcod=87'247)", 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.130388+0000", 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.38518406700000002, 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082803770000000002, 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.663 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130388+0000", 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130388+0000", 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4563000000000001e-05 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130423+0000", 2020-11-16T22:28:14.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9830000000000004e-06 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130428+0000", 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8660000000000006e-06 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130436+0000", 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5250000000000001e-06 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130439+0000", 2020-11-16T22:28:14.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8634999999999999e-05 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130478+0000", 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4630999999999998e-05 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.130502+0000", 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081097889999999992 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.138612+0000", 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6385000000000003e-05 2020-11-16T22:28:14.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.138669+0000", 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082803770000000002 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2244 4.7 4:ed2b824e:::10000000aef.00000000:head [write 0~744] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.246397+0000", 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.26917540299999998, 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080538759999999997, 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.671 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2244 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246397+0000", 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246397+0000", 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8401000000000001e-05 2020-11-16T22:28:14.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246425+0000", 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7339999999999996e-06 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246431+0000", 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1498000000000001e-05 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246443+0000", 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5219999999999998e-06 2020-11-16T22:28:14.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246448+0000", 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2480000000000001e-05 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246481+0000", 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011162699999999999 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246592+0000", 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011307999999999999 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.246705+0000", 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030548260000000001 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.249760+0000", 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044888020000000001 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.254249+0000", 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6166000000000002e-05 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.254325+0000", 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012574 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.254451+0000", 2020-11-16T22:28:14.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080538759999999997 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2246 4.7 4:ef08c8d3:::10000000af1.00000000:head [write 0~28] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.681 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.247734+0000", 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.26783835299999997, 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080146599999999998, 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.682 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2246 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247734+0000", 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247734+0000", 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.321e-05 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247787+0000", 2020-11-16T22:28:14.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.323e-06 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247794+0000", 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.456e-05 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247809+0000", 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.587e-06 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247815+0000", 2020-11-16T22:28:14.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2509999999999999e-05 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247847+0000", 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010787 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.247955+0000", 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011300500000000001 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.248068+0000", 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0046010269999999997 2020-11-16T22:28:14.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.252669+0000", 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028369839999999999 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.255506+0000", 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3466000000000003e-05 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.255599+0000", 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000149118 2020-11-16T22:28:14.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.255749+0000", 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080146599999999998 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.691 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2559 4.7 4:efb783ea:::10000000c32.00000000:head [write 0~3551] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.496858+0000", 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.018713924999999999, 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081027170000000006, 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:14.692 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2559 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496858+0000", 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496858+0000", 2020-11-16T22:28:14.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9224999999999999e-05 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496888+0000", 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0049999999999998e-06 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496892+0000", 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5870000000000004e-06 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496899+0000", 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2820000000000001e-06 2020-11-16T22:28:14.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496902+0000", 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6480000000000003e-05 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:14.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.496939+0000", 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7009999999999995e-05 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.497026+0000", 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9524000000000005e-05 2020-11-16T22:28:14.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.497125+0000", 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00053643300000000001 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:14.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.497662+0000", 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0071863140000000001 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.504848+0000", 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2434000000000001e-05 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.504891+0000", 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0423e-05 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.504961+0000", 2020-11-16T22:28:14.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0081027170000000006 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:14.702 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:14.704 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:14.704 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:14.717+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:14.804 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:14.818+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3081 4.2 4:46c4ccb5:::10000000e3b.00000000:head [write 0~43 in=43b] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.888544+0000", 2020-11-16T22:28:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0018236960000000001, 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0018424450000000001, 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.877 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3081 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888544+0000", 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888544+0000", 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9640000000000001e-05 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:14.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888574+0000", 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1790000000000001e-06 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888578+0000", 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4919999999999996e-06 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888585+0000", 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7589999999999999e-06 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888589+0000", 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2629e-05 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888642+0000", 2020-11-16T22:28:14.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014135499999999999 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.888783+0000", 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00023905399999999999 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3082 4.4 4:3d9650fb:::10000000e3c.00000000:head [write 0~2850 in=2850b] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.889297+0000", 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001071088, 2020-11-16T22:28:14.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001134564, 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.885 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3082 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889297+0000", 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:14.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889297+0000", 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7633000000000001e-05 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889324+0000", 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0960000000000003e-06 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:14.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889328+0000", 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2840000000000001e-06 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889336+0000", 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5549999999999998e-06 2020-11-16T22:28:14.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889339+0000", 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2366999999999997e-05 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889372+0000", 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0715000000000001e-05 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889462+0000", 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6764999999999999e-05 2020-11-16T22:28:14.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889559+0000", 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000262415 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.892 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3083 4.1 4:805eeac6:::10000000e3d.00000000:head [write 0~996 in=996b] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.889962+0000", 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00040537700000000001, 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00051083400000000005, 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:14.893 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3083 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:14.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889962+0000", 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889962+0000", 2020-11-16T22:28:14.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7413e-05 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889990+0000", 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6369999999999998e-06 2020-11-16T22:28:14.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.889994+0000", 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0560000000000005e-06 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:14.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.890002+0000", 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.048e-06 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.890006+0000", 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9689999999999999e-05 2020-11-16T22:28:14.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.890036+0000", 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5062000000000007e-05 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:14.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.890121+0000", 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000158906 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:28:14.900 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 3 2020-11-16T22:28:14.901 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:14.902 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:14.905 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:14.919+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:15.006 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:15.020+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:15.042 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:28:15.107 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:15.121+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:15.124 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:15.124 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:15.124 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:15.124 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:15.125 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:15.138 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:15.220 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:15.235+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:15.226 INFO:tasks.workunit.client.0.smithi138.stderr:Checking out files: 37% (1321/3521) Checking out files: 38% (1338/3521) Checking out files: 39% (1374/3521) Checking out files: 40% (1409/3521) Checking out files: 41% (1444/3521) Checking out files: 42% (1479/3521) Checking out files: 43% (1515/3521) Checking out files: 44% (1550/3521) Checking out files: 45% (1585/3521) Checking out files: 46% (1620/3521) Checking out files: 47% (1655/3521) Checking out files: 48% (1691/3521) Checking out files: 49% (1726/3521) Checking out files: 50% (1761/3521) Checking out files: 51% (1796/3521) Checking out files: 52% (1831/3521) Checking out files: 53% (1867/3521) Checking out files: 54% (1902/3521) Checking out files: 55% (1937/3521) Checking out files: 56% (1972/3521) Checking out files: 57% (2007/3521) Checking out files: 58% (2043/3521) Checking out files: 59% (2078/3521) Checking out files: 60% (2113/3521) Checking out files: 61% (2148/3521) Checking out files: 62% (2184/3521) Checking out files: 63% (2219/3521) Checking out files: 64% (2254/3521) Checking out files: 65% (2289/3521) Checking out files: 66% (2324/3521) Checking out files: 67% (2360/3521) Checking out files: 68% (2395/3521) Checking out files: 69% (2430/3521) Checking out files: 70% (2465/3521) Checking out files: 71% (2500/3521) Checking out files: 72% (2536/3521) Checking out files: 73% (2571/3521) Checking out files: 74% (2606/3521) Checking out files: 74% (2617/3521) Checking out files: 75% (2641/3521) Checking out files: 76% (2676/3521) Checking out files: 77% (2712/3521) Checking out files: 78% (2747/3521) Checking out files: 79% (2782/3521) Checking out files: 80% (2817/3521) Checking out files: 81% (2853/3521) Checking out files: 82% (2888/3521) Checking out files: 83% (2923/3521) Checking out files: 84% (2958/3521) Checking out files: 85% (2993/3521) Checking out files: 86% (3029/3521) Checking out files: 87% (3064/3521) Checking out files: 88% (3099/3521) Checking out files: 89% (3134/3521) Checking out files: 90% (3169/3521) Checking out files: 91% (3205/3521) Checking out files: 92% (3240/3521) Checking out files: 93% (3275/3521) Checking out files: 94% (3310/3521) Checking out files: 95% (3345/3521) Checking out files: 96% (3381/3521) Checking out files: 97% (3416/3521) Checking out files: 98% (3451/3521) Checking out files: 99% (3486/3521) Checking out files: 100% (3521/3521) Checking out files: 100% (3521/3521), done. 2020-11-16T22:28:15.235 INFO:tasks.workunit.client.0.smithi138.stderr:+ cd xfstests 2020-11-16T22:28:15.235 INFO:tasks.workunit.client.0.smithi138.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2020-11-16T22:28:15.308 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:15.322+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:15.320 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:15.320 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:15.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:15.320 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 10.010320645, 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:15.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:15.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:15.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:15.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:15.330 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 5.0098177570000004, 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.331 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:15.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:15.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:15.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:1756 4.3 4:c582dc3f:::10000000907.00000000:head [write 0~1479] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:13.872305+0000", 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.458591779, 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010683585000000001, 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.340 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 1756 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872305+0000", 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872305+0000", 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1551000000000002e-05 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872357+0000", 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1140000000000006e-06 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872365+0000", 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4076999999999999e-05 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872379+0000", 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1390000000000003e-06 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.872384+0000", 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001507031 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.873891+0000", 2020-11-16T22:28:15.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017637100000000001 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.874068+0000", 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0087527809999999994 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.882820+0000", 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4239000000000006e-05 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.882885+0000", 2020-11-16T22:28:15.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000104282 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.882989+0000", 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010683585000000001 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.348 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:1833 4.4 4:3d47e923:::10000000954.00000000:head [write 0~3543] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:13.927467+0000", 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.403430215, 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010436265, 2020-11-16T22:28:15.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 1833 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.350 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927467+0000", 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927467+0000", 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9110000000000001e-05 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927496+0000", 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8790000000000002e-06 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927501+0000", 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0873000000000001e-05 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927512+0000", 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1459999999999998e-06 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.927516+0000", 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003421197 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930937+0000", 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015707500000000001 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.931094+0000", 2020-11-16T22:28:15.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018151699999999999 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.931276+0000", 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029806590000000001 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.934256+0000", 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034854560000000001 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.937742+0000", 2020-11-16T22:28:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3119000000000002e-05 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.937815+0000", 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8233999999999996e-05 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.937903+0000", 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010436265 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:1838 4.2 4:46917829:::10000000959.00000000:head [write 0~1829] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:13.930879+0000", 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.4000185300000001, 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.01109548, 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.360 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 1838 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930879+0000", 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930879+0000", 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6350000000000002e-05 2020-11-16T22:28:15.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930925+0000", 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4309999999999996e-06 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930929+0000", 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0739999999999993e-06 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930937+0000", 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.976e-06 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930943+0000", 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9218000000000003e-05 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.930983+0000", 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016845799999999999 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.931151+0000", 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010726035 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.941877+0000", 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2602999999999999e-05 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.941920+0000", 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4335000000000001e-05 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:13.941974+0000", 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.01109548 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.368 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:2059 4.3 4:dc020d62:::10000000a36.00000000:head [write 0~1813] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.106995+0000", 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.2239020599999999, 2020-11-16T22:28:15.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010888488, 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 2059 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.370 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.106995+0000", 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.106995+0000", 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8200000000000001e-05 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.107023+0000", 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9360000000000001e-06 2020-11-16T22:28:15.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.107027+0000", 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3089999999999996e-06 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.107035+0000", 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1580000000000001e-06 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.107039+0000", 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037220980000000001 2020-11-16T22:28:15.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.110761+0000", 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000183735 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.110944+0000", 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067959040000000002 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.117740+0000", 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8357999999999998e-05 2020-11-16T22:28:15.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.117809+0000", 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4789999999999994e-05 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.117884+0000", 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010888488 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:2063 4.3 4:df0721c9:::10000000a3a.00000000:head [write 0~1879] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.378 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.109701+0000", 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.221195738, 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011043462, 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.379 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 2063 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109701+0000", 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109701+0000", 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7265000000000001e-05 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109729+0000", 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.532e-06 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109733+0000", 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4079999999999998e-06 2020-11-16T22:28:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109742+0000", 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3919999999999996e-06 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.109746+0000", 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017095369999999999 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.111455+0000", 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6009999999999998e-05 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.111542+0000", 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0090170589999999991 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.120559+0000", 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1977999999999995e-05 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.120621+0000", 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012428100000000001 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.120745+0000", 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011043462 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:111 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e87)", 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.211635+0000", 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.1192624019999999, 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.014698147, 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.388 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211635+0000", 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211635+0000", 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00024783500000000001 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211883+0000", 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4819999999999994e-06 2020-11-16T22:28:15.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211891+0000", 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9347e-05 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211910+0000", 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8969999999999996e-06 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211916+0000", 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7780000000000001e-05 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.211954+0000", 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5502999999999999e-05 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.212020+0000", 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094832709999999997 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.221503+0000", 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039774060000000002 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.225480+0000", 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00072345999999999999 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.226204+0000", 2020-11-16T22:28:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6397999999999998e-05 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.226260+0000", 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2768000000000005e-05 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.226333+0000", 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.014698147 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:2200 4.6 4:735ecdf0:::10000000ac3.00000000:head [write 0~318] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.212997+0000", 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.1179001200000001, 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.012432929000000001, 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.398 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 2200 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:15.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.212997+0000", 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.212997+0000", 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5549000000000005e-05 2020-11-16T22:28:15.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213063+0000", 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2830000000000004e-06 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:15.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213070+0000", 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3927000000000001e-05 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213084+0000", 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8520000000000004e-06 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213092+0000", 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8409999999999998e-05 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:15.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213150+0000", 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000110455 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213260+0000", 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00015268500000000001 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.213413+0000", 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012315749999999999 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.214645+0000", 2020-11-16T22:28:15.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010677749 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.225322+0000", 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1204000000000001e-05 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.225364+0000", 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6240000000000003e-05 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.225430+0000", 2020-11-16T22:28:15.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.012432929000000001 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:2794 4.2 4:592dfa2b:::10000000d1d.00000000:head [write 0~708] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.664662+0000", 2020-11-16T22:28:15.408 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.66623512399999996, 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.01239962, 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:15.409 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 2794 2020-11-16T22:28:15.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:15.410 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:19.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664662+0000", 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664662+0000", 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6028e-05 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664718+0000", 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3100000000000003e-06 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:19.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664725+0000", 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2364999999999999e-05 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664738+0000", 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.857e-06 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664744+0000", 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7113000000000002e-05 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664781+0000", 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000143177 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.664924+0000", 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011948947999999999 2020-11-16T22:28:19.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.676873+0000", 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1983000000000005e-05 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.676945+0000", 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116839 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.677062+0000", 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.01239962 2020-11-16T22:28:19.541 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:2795 4.4 4:261c7962:::10000000d1e.00000000:head [write 0~1684] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.665292+0000", 2020-11-16T22:28:19.542 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.66560548500000005, 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011760289, 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 2795 2020-11-16T22:28:19.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665292+0000", 2020-11-16T22:28:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665292+0000", 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8306999999999999e-05 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665320+0000", 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8109999999999997e-06 2020-11-16T22:28:19.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665330+0000", 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0789e-05 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665341+0000", 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9129999999999996e-06 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.665344+0000", 2020-11-16T22:28:19.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021913700000000002 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.667536+0000", 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010041500000000001 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:19.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.667636+0000", 2020-11-16T22:28:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00045521299999999999 2020-11-16T22:28:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.668091+0000", 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030158300000000002 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:19.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.671107+0000", 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0057529469999999996 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.676860+0000", 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5084000000000002e-05 2020-11-16T22:28:19.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.676925+0000", 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012661000000000001 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:19.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.677052+0000", 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011760289 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:19.958 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:122 3.3 3:ddb4e4d7:::200.00000002:head [write 402035~1472943 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e87)", 2020-11-16T22:28:19.959 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.774943+0000", 2020-11-16T22:28:19.959 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.55595437800000003, 2020-11-16T22:28:19.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.015029549999999999, 2020-11-16T22:28:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.774943+0000", 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.774943+0000", 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024570379999999999 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777400+0000", 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7370000000000003e-06 2020-11-16T22:28:20.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777405+0000", 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5480000000000007e-06 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777415+0000", 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777419+0000", 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1676e-05 2020-11-16T22:28:20.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777451+0000", 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6410999999999998e-05 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777518+0000", 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4346999999999996e-05 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.777612+0000", 2020-11-16T22:28:20.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011144391999999999 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.788756+0000", 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010786599999999999 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.789835+0000", 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8885000000000003e-05 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.789894+0000", 2020-11-16T22:28:20.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8405999999999994e-05 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.789972+0000", 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.015029549999999999 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.171 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.172 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3153 4.0 4:1734426d:::10000000e83.00000000:head [write 0~555] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:20.374 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.938852+0000", 2020-11-16T22:28:20.374 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.39204510399999998, 2020-11-16T22:28:20.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010825694, 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3153 2020-11-16T22:28:20.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938852+0000", 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938852+0000", 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0670000000000003e-05 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938883+0000", 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9579999999999998e-06 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938888+0000", 2020-11-16T22:28:20.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.146e-05 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938899+0000", 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4519999999999996e-06 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.938905+0000", 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00048730699999999997 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.939392+0000", 2020-11-16T22:28:20.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010705300000000001 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.939499+0000", 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012844700000000001 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.939627+0000", 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034310830000000001 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.943058+0000", 2020-11-16T22:28:20.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0063891349999999998 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.949448+0000", 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1396999999999993e-05 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.949529+0000", 2020-11-16T22:28:20.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148732 2020-11-16T22:28:20.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.949678+0000", 2020-11-16T22:28:20.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010825694 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3156 4.0 4:02302b6b:::10000000e86.00000000:head [write 0~2428] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:20.587 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.940892+0000", 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.39000464600000001, 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011915278, 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.588 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3156 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940892+0000", 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940892+0000", 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7705999999999999e-05 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940920+0000", 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0280000000000002e-06 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940924+0000", 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6580000000000008e-06 2020-11-16T22:28:20.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940933+0000", 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6500000000000004e-06 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.940937+0000", 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022095629999999999 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.943147+0000", 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016344800000000001 2020-11-16T22:28:20.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.943311+0000", 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012878 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.943439+0000", 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028164370000000002 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.946256+0000", 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0064094030000000001 2020-11-16T22:28:20.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.952665+0000", 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7788999999999998e-05 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.952723+0000", 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4815999999999996e-05 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.952808+0000", 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011915278 2020-11-16T22:28:20.597 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3162 4.0 4:0006d55c:::10000000e8c.00000000:head [write 0~2107] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:14.945343+0000", 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.385553851, 2020-11-16T22:28:20.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.014053394, 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3162 2020-11-16T22:28:20.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945343+0000", 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945343+0000", 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8906000000000001e-05 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945372+0000", 2020-11-16T22:28:20.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8239999999999999e-06 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945377+0000", 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1399999999999999e-05 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945388+0000", 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3929999999999999e-06 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.945394+0000", 2020-11-16T22:28:20.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044585459999999999 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.949852+0000", 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000108601 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.949961+0000", 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010331 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.950064+0000", 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030390980000000001 2020-11-16T22:28:20.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.953103+0000", 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0060907610000000001 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.959194+0000", 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4393000000000004e-05 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.959268+0000", 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000128162 2020-11-16T22:28:20.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:14.959397+0000", 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.014053394 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3354 4.6 4:70d51104:::10000000f4c.00000000:head [write 0~990] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:15.084781+0000", 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.246116214, 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011200646999999999, 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.609 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3354 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084781+0000", 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084781+0000", 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7772000000000001e-05 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084809+0000", 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8969999999999999e-06 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084814+0000", 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4009999999999997e-06 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084822+0000", 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1640000000000003e-06 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084826+0000", 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5932000000000001e-05 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084862+0000", 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2899000000000006e-05 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.084955+0000", 2020-11-16T22:28:20.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000104079 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085059+0000", 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0020412749999999999 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.087100+0000", 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0086568879999999997 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.095757+0000", 2020-11-16T22:28:20.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9958999999999997e-05 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.095847+0000", 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013438100000000001 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.095982+0000", 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011200646999999999 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3355 4.6 4:78ada98e:::10000000f4d.00000000:head [write 0~82] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:15.085439+0000", 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.245458496, 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010745579, 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3355 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085439+0000", 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085439+0000", 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9023e-05 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085468+0000", 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0390000000000004e-06 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085473+0000", 2020-11-16T22:28:20.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7579999999999998e-06 2020-11-16T22:28:20.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085481+0000", 2020-11-16T22:28:20.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3470000000000001e-06 2020-11-16T22:28:20.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.085486+0000", 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00091695700000000004 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.086403+0000", 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016318800000000001 2020-11-16T22:28:20.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.086566+0000", 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103321 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.086669+0000", 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011640000000000001 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.087833+0000", 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0081622510000000006 2020-11-16T22:28:20.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.095995+0000", 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7267000000000005e-05 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.096063+0000", 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000121428 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.096184+0000", 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010745579 2020-11-16T22:28:20.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3365 4.6 4:6b88f81e:::10000000f57.00000000:head [write 0~85] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:15.092111+0000", 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.23878569399999999, 2020-11-16T22:28:20.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010415839, 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3365 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.833 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092111+0000", 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092111+0000", 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8087999999999998e-05 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092159+0000", 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4989999999999997e-06 2020-11-16T22:28:20.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092167+0000", 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4416999999999999e-05 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:20.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092181+0000", 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7910000000000001e-06 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.092187+0000", 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040247900000000003 2020-11-16T22:28:20.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.096212+0000", 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2099e-05 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:20.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.096304+0000", 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5556000000000005e-05 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.096400+0000", 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026858659999999999 2020-11-16T22:28:20.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.099086+0000", 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0032727619999999998 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:20.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.102358+0000", 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8245999999999997e-05 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.102427+0000", 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000100725 2020-11-16T22:28:20.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.102527+0000", 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010415839 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:20.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3367 4.6 4:6ed46414:::10000000f59.00000000:head [write 0~124] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:15.093442+0000", 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.23745521999999999, 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.012523069, 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:20.843 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3367 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:20.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093442+0000", 2020-11-16T22:28:20.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:20.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:20.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:20.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:21.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093442+0000", 2020-11-16T22:28:21.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6579000000000001e-05 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093468+0000", 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1729999999999997e-06 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093473+0000", 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4599999999999997e-06 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093480+0000", 2020-11-16T22:28:21.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.366e-06 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.093483+0000", 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0056195159999999997 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.099103+0000", 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1335999999999994e-05 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.099194+0000", 2020-11-16T22:28:21.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010376 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.099298+0000", 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0013022470000000001 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.100600+0000", 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051323289999999997 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.105733+0000", 2020-11-16T22:28:21.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0463999999999995e-05 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.105803+0000", 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016183899999999999 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.105965+0000", 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.012523069 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:3379 4.2 4:5243d446:::10000000f65.00000000:head [write 0~44] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:15.101606+0000", 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.229291146, 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011233251, 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:21.055 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 3379 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:21.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101606+0000", 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101606+0000", 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6539000000000001e-05 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101632+0000", 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.331e-06 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:21.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101636+0000", 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2210000000000002e-06 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101642+0000", 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6819999999999999e-06 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101645+0000", 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1069999999999999e-05 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:21.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101676+0000", 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.9241999999999996e-05 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.101775+0000", 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010919808 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.112695+0000", 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1017999999999999e-05 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:21.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.112736+0000", 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010334 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:15.112839+0000", 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011233251 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:21.063 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:21.064 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:21.064 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:21.064 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:21.064 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:21.065 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:21.065 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr:Note: checking out '12973fc04fd10d4af086901e10ffa8e48866b735'. 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr:state without impacting any branches by performing another checkout. 2020-11-16T22:28:21.072 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr:do so (now or later) by using -b with the checkout command again. Example: 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr: git checkout -b 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr: 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2020-11-16T22:28:21.073 INFO:tasks.workunit.client.0.smithi138.stderr:+ make -j4 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize -cfi 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: putting auxiliary files in '.'. 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file './config.guess' 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file './config.sub' 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file './install-sh' 2020-11-16T22:28:21.074 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file './ltmain.sh' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file 'm4/libtool.m4' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file 'm4/ltversion.m4' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2020-11-16T22:28:21.075 INFO:tasks.workunit.client.0.smithi138.stdout:cp include/install-sh . 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:aclocal -I m4 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:autoheader 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:autoconf 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:./configure \ 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout: --libexecdir=/usr/lib \ 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout: --exec_prefix=/var/lib 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:checking build system type... x86_64-pc-linux-gnu 2020-11-16T22:28:21.076 INFO:tasks.workunit.client.0.smithi138.stdout:checking host system type... x86_64-pc-linux-gnu 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to print strings... printf 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gcc... gcc 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether the C compiler works... yes 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking for C compiler default output file name... a.out 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking for suffix of executables... 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether we are cross compiling... no 2020-11-16T22:28:21.077 INFO:tasks.workunit.client.0.smithi138.stdout:checking for suffix of object files... o 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether we are using the GNU C compiler... yes 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether gcc accepts -g... yes 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gcc option to accept ISO C89... none needed 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for egrep... /usr/bin/grep -E 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for fgrep... /usr/bin/grep -F 2020-11-16T22:28:21.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ld used by gcc... /usr/bin/ld 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether ln -s works... yes 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking the maximum length of command line arguments... 1572864 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2020-11-16T22:28:21.079 INFO:tasks.workunit.client.0.smithi138.stdout:checking for /usr/bin/ld option to reload object files... -r 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking for objdump... objdump 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to recognize dependent libraries... pass_all 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking for dlltool... no 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to associate runtime and link libraries... printf %s\n 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ar... ar 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking for archiver @FILE support... @ 2020-11-16T22:28:21.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking for strip... strip 2020-11-16T22:28:21.081 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ranlib... ranlib 2020-11-16T22:28:21.081 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gawk... gawk 2020-11-16T22:28:21.081 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.080+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.082 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:21.144 INFO:tasks.workunit.client.0.smithi138.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2020-11-16T22:28:21.145 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sysroot... no 2020-11-16T22:28:21.154 INFO:tasks.workunit.client.0.smithi138.stdout:checking for a working dd... /usr/bin/dd 2020-11-16T22:28:21.161 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2020-11-16T22:28:21.166 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.181+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.187 INFO:tasks.workunit.client.0.smithi138.stdout:checking for mt... no 2020-11-16T22:28:21.195 INFO:tasks.workunit.client.0.smithi138.stdout:checking if : is a manifest tool... no 2020-11-16T22:28:21.213 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:21.213 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:21.214 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:21.214 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:21.223 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:21.246 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:21.262 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to run the C preprocessor... gcc -E 2020-11-16T22:28:21.267 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:21.281+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:21.273 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:21.276 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:21.276 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:21.350 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:21.350 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:21.361 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:21.402 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:21.416+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:21.459 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ANSI C header files... yes 2020-11-16T22:28:21.469 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:21.483+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:21.489 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/types.h... yes 2020-11-16T22:28:21.521 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/stat.h... yes 2020-11-16T22:28:21.533 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:21.534 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:21.544 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:21.557 INFO:tasks.workunit.client.0.smithi138.stdout:checking for stdlib.h... yes 2020-11-16T22:28:21.587 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.600+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.591 INFO:tasks.workunit.client.0.smithi138.stdout:checking for string.h... yes 2020-11-16T22:28:21.626 INFO:tasks.workunit.client.0.smithi138.stdout:checking for memory.h... yes 2020-11-16T22:28:21.659 INFO:tasks.workunit.client.0.smithi138.stdout:checking for strings.h... yes 2020-11-16T22:28:21.670 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.683+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.674 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:21.675 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:21.684 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:21.711 INFO:tasks.workunit.client.0.smithi138.stdout:checking for inttypes.h... yes 2020-11-16T22:28:21.744 INFO:tasks.workunit.client.0.smithi138.stdout:checking for stdint.h... yes 2020-11-16T22:28:21.771 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:21.785+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:21.780 INFO:tasks.workunit.client.0.smithi138.stdout:checking for unistd.h... yes 2020-11-16T22:28:21.823 INFO:tasks.workunit.client.0.smithi138.stdout:checking for dlfcn.h... yes 2020-11-16T22:28:21.828 INFO:tasks.workunit.client.0.smithi138.stdout:checking for objdir... .libs 2020-11-16T22:28:21.854 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:21.855 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:21.864 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:21.872 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2020-11-16T22:28:21.907 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.921+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.927 INFO:tasks.workunit.client.0.smithi138.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2020-11-16T22:28:21.928 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2020-11-16T22:28:21.954 INFO:tasks.workunit.client.0.smithi138.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2020-11-16T22:28:21.973 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:21.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:21.993 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:21.993 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:21.999 INFO:tasks.workunit.client.0.smithi138.stdout:checking if gcc static flag -static works... no 2020-11-16T22:28:22.003 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:22.064 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:28:22.073 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.087+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.080 INFO:tasks.workunit.client.0.smithi138.stdout:checking if gcc supports -c -o file.o... yes 2020-11-16T22:28:22.081 INFO:tasks.workunit.client.0.smithi138.stdout:checking if gcc supports -c -o file.o... (cached) yes 2020-11-16T22:28:22.121 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2020-11-16T22:28:22.136 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.136 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.145 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:22.164 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether -lc should be explicitly linked in... no 2020-11-16T22:28:22.186 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.200+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.224 INFO:tasks.workunit.client.0.smithi138.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2020-11-16T22:28:22.225 INFO:tasks.workunit.client.0.smithi138.stdout:checking how to hardcode library paths into programs... immediate 2020-11-16T22:28:22.226 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether stripping libraries is possible... yes 2020-11-16T22:28:22.226 INFO:tasks.workunit.client.0.smithi138.stdout:checking if libtool supports shared libraries... yes 2020-11-16T22:28:22.226 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether to build shared libraries... yes 2020-11-16T22:28:22.227 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether to build static libraries... yes 2020-11-16T22:28:22.231 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gcc... (cached) gcc 2020-11-16T22:28:22.257 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether we are using the GNU C compiler... (cached) yes 2020-11-16T22:28:22.257 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether gcc accepts -g... (cached) yes 2020-11-16T22:28:22.258 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2020-11-16T22:28:22.258 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gmake... /usr/bin/gmake 2020-11-16T22:28:22.258 INFO:tasks.workunit.client.0.smithi138.stderr:./configure: line 11956: test: $(SHELL): binary operator expected 2020-11-16T22:28:22.259 INFO:tasks.workunit.client.0.smithi138.stdout:checking for tar... /bin/tar 2020-11-16T22:28:22.259 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gzip... /bin/gzip 2020-11-16T22:28:22.268 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether gcc -MM is supported... yes 2020-11-16T22:28:22.269 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sort... /bin/sort 2020-11-16T22:28:22.269 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether ln -s works... yes 2020-11-16T22:28:22.270 INFO:tasks.workunit.client.0.smithi138.stdout:checking for rpm... /bin/rpm 2020-11-16T22:28:22.273 INFO:tasks.workunit.client.0.smithi138.stdout:checking for rpmbuild... no 2020-11-16T22:28:22.274 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ANSI C header files... (cached) yes 2020-11-16T22:28:22.276 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.290+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.309 INFO:tasks.workunit.client.0.smithi138.stdout:checking assert.h usability... yes 2020-11-16T22:28:22.314 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.315 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.319 INFO:tasks.workunit.client.0.smithi138.stdout:checking assert.h presence... yes 2020-11-16T22:28:22.320 INFO:tasks.workunit.client.0.smithi138.stdout:checking for assert.h... yes 2020-11-16T22:28:22.324 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:22.361 INFO:tasks.workunit.client.0.smithi138.stdout:checking bstring.h usability... no 2020-11-16T22:28:22.379 INFO:tasks.workunit.client.0.smithi138.stdout:checking bstring.h presence... no 2020-11-16T22:28:22.380 INFO:tasks.workunit.client.0.smithi138.stdout:checking for bstring.h... no 2020-11-16T22:28:22.407 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.421+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.413 INFO:tasks.workunit.client.0.smithi138.stdout:checking libgen.h usability... yes 2020-11-16T22:28:22.423 INFO:tasks.workunit.client.0.smithi138.stdout:checking libgen.h presence... yes 2020-11-16T22:28:22.424 INFO:tasks.workunit.client.0.smithi138.stdout:checking for libgen.h... yes 2020-11-16T22:28:22.462 INFO:tasks.workunit.client.0.smithi138.stdout:checking dirent.h usability... yes 2020-11-16T22:28:22.473 INFO:tasks.workunit.client.0.smithi138.stdout:checking dirent.h presence... yes 2020-11-16T22:28:22.473 INFO:tasks.workunit.client.0.smithi138.stdout:checking for dirent.h... yes 2020-11-16T22:28:22.477 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:22.490+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:22.493 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.493 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.502 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:22.509 INFO:tasks.workunit.client.0.smithi138.stdout:checking errno.h usability... yes 2020-11-16T22:28:22.519 INFO:tasks.workunit.client.0.smithi138.stdout:checking errno.h presence... yes 2020-11-16T22:28:22.520 INFO:tasks.workunit.client.0.smithi138.stdout:checking for errno.h... yes 2020-11-16T22:28:22.558 INFO:tasks.workunit.client.0.smithi138.stdout:checking malloc.h usability... yes 2020-11-16T22:28:22.570 INFO:tasks.workunit.client.0.smithi138.stdout:checking malloc.h presence... yes 2020-11-16T22:28:22.570 INFO:tasks.workunit.client.0.smithi138.stdout:checking for malloc.h... yes 2020-11-16T22:28:22.578 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.592+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.609 INFO:tasks.workunit.client.0.smithi138.stdout:checking uuid.h usability... no 2020-11-16T22:28:22.628 INFO:tasks.workunit.client.0.smithi138.stdout:checking uuid.h presence... no 2020-11-16T22:28:22.628 INFO:tasks.workunit.client.0.smithi138.stdout:checking for uuid.h... no 2020-11-16T22:28:22.647 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.647 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.657 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:22.662 INFO:tasks.workunit.client.0.smithi138.stdout:checking uuid/uuid.h usability... yes 2020-11-16T22:28:22.674 INFO:tasks.workunit.client.0.smithi138.stdout:checking uuid/uuid.h presence... yes 2020-11-16T22:28:22.674 INFO:tasks.workunit.client.0.smithi138.stdout:checking for uuid/uuid.h... yes 2020-11-16T22:28:22.679 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.693+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.714 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/uuid.h usability... no 2020-11-16T22:28:22.733 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/uuid.h presence... no 2020-11-16T22:28:22.733 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/uuid.h... no 2020-11-16T22:28:22.767 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/file.h usability... yes 2020-11-16T22:28:22.778 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/file.h presence... yes 2020-11-16T22:28:22.778 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/file.h... yes 2020-11-16T22:28:22.780 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:22.794+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:22.790 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.790 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.799 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:22.816 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fcntl.h usability... yes 2020-11-16T22:28:22.827 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fcntl.h presence... yes 2020-11-16T22:28:22.827 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/fcntl.h... yes 2020-11-16T22:28:22.867 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/syssgi.h usability... no 2020-11-16T22:28:22.881 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:22.894+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:22.888 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/syssgi.h presence... no 2020-11-16T22:28:22.888 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/syssgi.h... no 2020-11-16T22:28:22.928 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:22.928 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:22.937 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:22.957 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/param.h usability... yes 2020-11-16T22:28:22.971 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/param.h presence... yes 2020-11-16T22:28:22.971 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/param.h... yes 2020-11-16T22:28:22.980 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/stat.h... (cached) yes 2020-11-16T22:28:22.982 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:22.996+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:23.014 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/statvfs.h usability... yes 2020-11-16T22:28:23.027 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/statvfs.h presence... yes 2020-11-16T22:28:23.027 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/statvfs.h... yes 2020-11-16T22:28:23.069 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:23.069 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:23.078 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/time.h usability... yes 2020-11-16T22:28:23.079 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:23.089 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/time.h presence... yes 2020-11-16T22:28:23.090 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/time.h... yes 2020-11-16T22:28:23.123 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:23.137+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:23.126 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/ioctl.h usability... yes 2020-11-16T22:28:23.137 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/ioctl.h presence... yes 2020-11-16T22:28:23.137 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/ioctl.h... yes 2020-11-16T22:28:23.178 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/wait.h usability... yes 2020-11-16T22:28:23.184 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:23.198+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:23.190 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/wait.h presence... yes 2020-11-16T22:28:23.191 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/wait.h... yes 2020-11-16T22:28:23.203 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/types.h... (cached) yes 2020-11-16T22:28:23.207 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:23.207 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:23.208 INFO:tasks.workunit.client.0.smithi138.stdout:checking for strings.h... (cached) yes 2020-11-16T22:28:23.217 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:23.246 INFO:tasks.workunit.client.0.smithi138.stdout:checking err.h usability... yes 2020-11-16T22:28:23.257 INFO:tasks.workunit.client.0.smithi138.stdout:checking err.h presence... yes 2020-11-16T22:28:23.257 INFO:tasks.workunit.client.0.smithi138.stdout:checking for err.h... yes 2020-11-16T22:28:23.278 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:23.285 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:23.299+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:23.294 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/falloc.h usability... yes 2020-11-16T22:28:23.305 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/falloc.h presence... yes 2020-11-16T22:28:23.305 INFO:tasks.workunit.client.0.smithi138.stdout:checking for linux/falloc.h... yes 2020-11-16T22:28:23.346 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fs/xfs_fsops.h usability... no 2020-11-16T22:28:23.365 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fs/xfs_fsops.h presence... no 2020-11-16T22:28:23.366 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/fs/xfs_fsops.h... no 2020-11-16T22:28:23.385 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:23.400+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:23.386 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:23.387 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:23.397 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:23.402 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fs/xfs_itable.h usability... no 2020-11-16T22:28:23.423 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/fs/xfs_itable.h presence... no 2020-11-16T22:28:23.424 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/fs/xfs_itable.h... no 2020-11-16T22:28:23.432 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:23.453 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:23.453 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:23.453 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:23.454 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:23.456 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:23.456 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:23.459 INFO:tasks.workunit.client.0.smithi138.stdout:checking xfs/platform_defs.h usability... no 2020-11-16T22:28:23.478 INFO:tasks.workunit.client.0.smithi138.stdout:checking xfs/platform_defs.h presence... no 2020-11-16T22:28:23.479 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/platform_defs.h... no 2020-11-16T22:28:23.486 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:23.500+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:23.514 INFO:tasks.workunit.client.0.smithi138.stdout:checking btrfs/ioctl.h usability... no 2020-11-16T22:28:23.535 INFO:tasks.workunit.client.0.smithi138.stdout:checking btrfs/ioctl.h presence... no 2020-11-16T22:28:23.536 INFO:tasks.workunit.client.0.smithi138.stdout:checking for btrfs/ioctl.h... no 2020-11-16T22:28:23.546 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.546 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:2064 4.0 4:19f45ccf:::1000000166a.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e89)", 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:23.557211+0000", 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0036326079999999998, 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036515860000000001, 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557211+0000", 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557211+0000", 2020-11-16T22:28:23.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5848999999999999e-05 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557237+0000", 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5499999999999996e-06 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557241+0000", 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.072e-06 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557249+0000", 2020-11-16T22:28:23.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6200000000000001e-06 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557253+0000", 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3059000000000003e-05 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557286+0000", 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3857999999999996e-05 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557370+0000", 2020-11-16T22:28:23.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012488500000000001 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.557495+0000", 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00107551 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.558570+0000", 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0013594029999999999 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.554 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:2067 4.4 4:215730d6:::10000001667.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e89)", 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:23.559537+0000", 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0013061170000000001, 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001384115, 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559537+0000", 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559537+0000", 2020-11-16T22:28:23.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7758000000000001e-05 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559565+0000", 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8430000000000004e-06 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559572+0000", 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0000000000000001e-05 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559582+0000", 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3590000000000001e-06 2020-11-16T22:28:23.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559586+0000", 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1163e-05 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559617+0000", 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0836999999999998e-05 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559708+0000", 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1624000000000004e-05 2020-11-16T22:28:23.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559780+0000", 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00080783799999999996 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:23.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560588+0000", 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010504220000000001 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:2068 4.0 4:0d1c1fb3:::10000001666.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force e89)", 2020-11-16T22:28:23.562 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:23.559812+0000", 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0010315420000000001, 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001180457, 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559812+0000", 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559812+0000", 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5573000000000001e-05 2020-11-16T22:28:23.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559837+0000", 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6770000000000003e-06 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559842+0000", 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1219999999999995e-06 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559850+0000", 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.36e-06 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559854+0000", 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7817000000000001e-05 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559891+0000", 2020-11-16T22:28:23.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6788999999999998e-05 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.559988+0000", 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012886000000000001 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560117+0000", 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00030519800000000001 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:5349 4.0 4:18ac90e8:::10000001729.00000000:head [write 0~1625 in=1625b] snapc 1=[] ondisk+write+known_if_redirected e89)", 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:23.560663+0000", 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00018060400000000001, 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037995699999999999, 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:23.570 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 5349 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560663+0000", 2020-11-16T22:28:23.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560663+0000", 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8235e-05 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560691+0000", 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.583e-06 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560696+0000", 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2299999999999997e-06 2020-11-16T22:28:23.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560705+0000", 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7670000000000001e-06 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560709+0000", 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2638999999999997e-05 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560741+0000", 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102867 2020-11-16T22:28:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560844+0000", 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013198300000000001 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:23.560976+0000", 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00031330399999999998 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 4 2020-11-16T22:28:23.577 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:23.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking cifs/ioctl.h usability... no 2020-11-16T22:28:23.579 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:23.587 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:23.601+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:23.600 INFO:tasks.workunit.client.0.smithi138.stdout:checking cifs/ioctl.h presence... no 2020-11-16T22:28:23.600 INFO:tasks.workunit.client.0.smithi138.stdout:checking for cifs/ioctl.h... no 2020-11-16T22:28:23.637 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/mman.h usability... yes 2020-11-16T22:28:23.648 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/mman.h presence... yes 2020-11-16T22:28:23.648 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/mman.h... yes 2020-11-16T22:28:23.678 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/xfs_log_format.h... no 2020-11-16T22:28:23.688 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:23.701+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:23.733 INFO:tasks.workunit.client.0.smithi138.stdout:checking for uuid_compare... no 2020-11-16T22:28:23.748 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:23.748 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:23.749 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:23.749 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:23.749 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:23.758 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:23.777 INFO:tasks.workunit.client.0.smithi138.stdout:checking for uuid_compare in -luuid... yes 2020-11-16T22:28:23.788 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:23.803+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:23.820 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/xfs.h... yes 2020-11-16T22:28:23.843 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/libxfs.h... no 2020-11-16T22:28:23.867 INFO:tasks.workunit.client.0.smithi138.stdout:checking whether xlog_assign_lsn is declared... no 2020-11-16T22:28:23.889 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:23.902+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:23.898 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/xqm.h... yes 2020-11-16T22:28:23.931 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:23.931 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:23.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:23.931 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 18.623722536999999, 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:23.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:23.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:23.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:23.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:23.941 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.623219648999999, 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:23.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:23.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:23.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:23.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:23.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:23.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 7.694548417, 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:24.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:24.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:24.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:24.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:24.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:24.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:24.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:24.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:24.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 6.9772619459999996, 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:24.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:24.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:24.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:24.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:24.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 6.4344246360000001, 2020-11-16T22:28:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:24.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:24.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:24.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:24.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:25.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:25.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:25.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.199 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:25.199 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:25.199 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7330997330000004, 2020-11-16T22:28:25.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:25.402 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:25.402 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:25.402 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:25.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:25.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:25.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:25.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:25.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:25.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.614 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:25.614 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:25.614 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7325687199999997, 2020-11-16T22:28:25.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:25.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:25.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:25.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:25.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:25.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:25.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:25.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:25.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:25.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:25.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:26.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:26.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.030 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7324478919999997, 2020-11-16T22:28:26.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:26.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:26.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:26.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:26.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.243 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7323655779999996, 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:26.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:26.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:26.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:26.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.253 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7322729490000004, 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:26.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:26.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:26.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:26.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:26.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:26.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7321906609999997, 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.465 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:26.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:26.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:26.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:26.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7321083169999998, 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.475 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:26.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:26.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.484 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7320331969999998, 2020-11-16T22:28:26.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:26.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:26.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:26.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:26.494 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:26.495 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7319542630000004, 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:26.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:26.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:26.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7318768410000001, 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:26.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:26.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.515 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7317966790000003, 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:26.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:26.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:26.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:26.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.525 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7317148659999999, 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:26.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:26.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:26.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:26.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7316214800000003, 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:26.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:26.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:26.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.545 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7315010839999996, 2020-11-16T22:28:26.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:26.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:26.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.555 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.7314010169999996, 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:26.556 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:26.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:26.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:26.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:26.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:26.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:26.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:26.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:26.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:26.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:26.567 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:26.567 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:26.567 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:28:26.568 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:26.576 INFO:tasks.workunit.client.0.smithi138.stdout:checking xfsctl from xfs/xfs.h... ok 2020-11-16T22:28:26.576 INFO:tasks.workunit.client.0.smithi138.stdout:checking xfs/handle.h usability... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking xfs/handle.h presence... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking for xfs/handle.h... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking for attr_list_by_handle in -lhandle... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking attr/attributes.h usability... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking attr/attributes.h presence... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking for attr/attributes.h... yes 2020-11-16T22:28:26.577 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/attributes.h usability... no 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/attributes.h presence... no 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/attributes.h... no 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking for attr_list in -lattr... yes 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking for attr_set in -lattr... yes 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/acl.h usability... yes 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/acl.h presence... yes 2020-11-16T22:28:26.578 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/acl.h... yes 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking acl/libacl.h usability... yes 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking acl/libacl.h presence... yes 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking for acl/libacl.h... yes 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking for acl_init in -lacl... yes 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking gdbm-ndbm.h usability... no 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking gdbm-ndbm.h presence... no 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gdbm-ndbm.h... no 2020-11-16T22:28:26.579 INFO:tasks.workunit.client.0.smithi138.stdout:checking gdbm/ndbm.h usability... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking gdbm/ndbm.h presence... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking for gdbm/ndbm.h... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking ndbm.h usability... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking ndbm.h presence... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking for ndbm.h... no 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking libaio.h usability... yes 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking libaio.h presence... yes 2020-11-16T22:28:26.580 INFO:tasks.workunit.client.0.smithi138.stdout:checking for libaio.h... yes 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout:checking for dm_make_handle in -ldm... no 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout: 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout:WARNING: could not find a valid DMAPI base library. 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout:If you want DMAPI support please install the data migration 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout:API (dmapi) library package. Alternatively, run "make install" 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout:from the dmapi source. 2020-11-16T22:28:26.581 INFO:tasks.workunit.client.0.smithi138.stdout: 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/fiemap.h usability... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/fiemap.h presence... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking for linux/fiemap.h... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking for fallocate... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking for open_by_handle_at... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/prctl.h usability... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking sys/prctl.h presence... yes 2020-11-16T22:28:26.582 INFO:tasks.workunit.client.0.smithi138.stdout:checking for sys/prctl.h... yes 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/fs.h usability... yes 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking linux/fs.h presence... yes 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking for linux/fs.h... yes 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking btrfsutil.h usability... no 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking btrfsutil.h presence... no 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking for btrfsutil.h... no 2020-11-16T22:28:26.583 INFO:tasks.workunit.client.0.smithi138.stdout:checking for copy_file_range... yes 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:checking for renameat2... yes 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:configure: creating ./config.status 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:config.status: creating include/builddefs 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:config.status: creating include/config.h 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:config.status: executing libtool commands 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:Building include 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:Building lib 2020-11-16T22:28:26.584 INFO:tasks.workunit.client.0.smithi138.stdout:Building tests 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [LTDEP] 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dataascii.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] databin.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] datapid.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] file_lock.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] forker.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] pattern.lo 2020-11-16T22:28:26.585 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] open_flags.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] random_range.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] string_to_tokens.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] str_to_bytes.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] tlibio.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] write_log.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] random.lo 2020-11-16T22:28:26.586 INFO:tasks.workunit.client.0.smithi138.stdout: [LD] 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout:Building ltp 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout:Building src 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout:Building m4 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout:Building common 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout: [DEP] 2020-11-16T22:28:26.587 INFO:tasks.workunit.client.0.smithi138.stdout: [DEP] 2020-11-16T22:28:26.588 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:26.581+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:26.588 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:26.669 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:26.683+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:26.872 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:26.872 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:26.872 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:26.872 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:26.873 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:26.873 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:26.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:26.873 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:26.873 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:26.874 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:26.874 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:26.874 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:26.874 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:26.885+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:26.875 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:26.967 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] doio 2020-11-16T22:28:26.968 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fsstress 2020-11-16T22:28:26.968 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fsx 2020-11-16T22:28:26.972 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:26.986+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:27.038 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.038 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.048 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:27.073 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.087+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.089 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:28:27.174 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.188+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.213 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.213 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.222 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:27.236 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c: In function 'attr_list_path': 2020-11-16T22:28:27.236 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c:859:2: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr: rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~ 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr:In file included from ../src/global.h:25, 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr: from fsstress.c:12: 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr: extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2020-11-16T22:28:27.237 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~ 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c: In function 'attr_remove_path': 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c:878:2: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr: rval = attr_remove(name->path, attrname, flags); 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~ 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr:In file included from ../src/global.h:25, 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr: from fsstress.c:12: 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2020-11-16T22:28:27.238 INFO:tasks.workunit.client.0.smithi138.stderr: extern int attr_remove (const char *__path, const char *__attrname, 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~ 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c: In function 'attr_set_path': 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr:fsstress.c:898:2: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr: rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~ 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr:In file included from ../src/global.h:25, 2020-11-16T22:28:27.239 INFO:tasks.workunit.client.0.smithi138.stderr: from fsstress.c:12: 2020-11-16T22:28:27.240 INFO:tasks.workunit.client.0.smithi138.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2020-11-16T22:28:27.240 INFO:tasks.workunit.client.0.smithi138.stderr: extern int attr_set (const char *__path, const char *__attrname, 2020-11-16T22:28:27.240 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~ 2020-11-16T22:28:27.275 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:27.289+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:27.349 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.350 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.358 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:27.376 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:27.390+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:27.476 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.490+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.483 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.484 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.492 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:27.577 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:27.592+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:27.657 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.658 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.666 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:27.678 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.691+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2020-11-16T22:28:27.770 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] growfiles 2020-11-16T22:28:27.779 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.793+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.794 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.794 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.803 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:27.817 INFO:tasks.workunit.client.0.smithi138.stderr:fsx.c: In function 'main': 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr:fsx.c:2662:5: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr: strncpy(opsfile, optarg, sizeof(opsfile)); 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr:fsx.c:2632:4: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr: strncpy(dname, optarg, sizeof(dname)); 2020-11-16T22:28:27.818 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:27.879 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:27.893+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:27.928 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:27.928 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:27.937 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:27.941 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] iogen 2020-11-16T22:28:27.982 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:27.996+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:28.063 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.063 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.072 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:28.113 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:28.127+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:28.181 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.195+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.197 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.197 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.207 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:28.289 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.303+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.371 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.371 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.379 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:28.412 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-stress 2020-11-16T22:28:28.421 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.434+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.484 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.497+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.546 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.546 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.555 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:28.597 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.610+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.683 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.683 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.685 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:28.699+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:28.692 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:28.786 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:28.800+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:28.857 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:28.857 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:28.866 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:28.874 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:28.886 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:28.901+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:28.988 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:29.002+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:29.031 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:29.033 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:29.034 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 311.53824773999997, 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:29.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:29.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:29.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:29.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:29.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:29.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:29.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.247 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 311.53813109700002, 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:29.248 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:29.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:29.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:29.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:29.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:29.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:29.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:29.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:29.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 311.53798272799997, 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:29.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:29.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:29.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:29.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:29.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:29.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:29.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:29.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:29.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:29.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:29.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.880 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:29.880 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 311.53719929900001, 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:29.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:29.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:30.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:30.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:30.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:30.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:30.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:30.092 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 14.981162871, 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:30.093 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:30.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:30.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:30.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:30.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:30.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:30.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:30.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:30.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:30.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:30.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:30.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 14.268893381, 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:28:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:30.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:28:30.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:28:30.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.792466685999999, 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:30.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:28:30.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:30.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:28:30.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:30.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:28:30.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:30.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.727 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:30.728 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:30.728 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.722244221, 2020-11-16T22:28:30.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:28:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:30.930 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.931 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:28:30.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.933 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:28:30.934 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:28:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:30.935 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:28:31.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:28:31.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:28:31.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:28:31.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.072308489999999, 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.144 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:28:31.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:28:31.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.151 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.066561181999999, 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:28:31.152 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.153 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:28:31.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.065425178, 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.163 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:28:31.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:28:31.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:28:31.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:28:31.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:28:31.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:28:31.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.064278082, 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.377 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:28:31.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:28:31.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:28:31.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.387 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.058336710000001, 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.388 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:28:31.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:28:31.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.055991970999999, 2020-11-16T22:28:31.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.399 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:28:31.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.610 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.610 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:31.610 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.051245833999999, 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:31.611 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.613 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:28:31.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:28:31.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:31.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:28:31.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:28:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 11.532379291, 2020-11-16T22:28:31.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:31.622 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:28:31.624 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.625 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:28:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.627 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:28:31.628 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:31.629 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 11.013362525, 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:28:31.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:28:31.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:28:31.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:31.637 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 9.8365553559999999, 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:28:31.645 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 9.8198631540000001, 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:28:31.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:28:31.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:28:31.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.852 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 8.7906775780000004, 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:31.855 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:28:31.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:28:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.860 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:31.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:28:31.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:31.863 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:31.864 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:28:31.868 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:31.868 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:31.868 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:31.868 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:31.869 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:31.869 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:31.869 INFO:tasks.workunit.client.0.smithi138.stdout: [DEP] 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [DEP] 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [DEP] 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dirstress 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fill2 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fill 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] getpagesize 2020-11-16T22:28:31.870 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] holes 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] lstat64 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] nametest 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] permname 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] randholes 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] runas 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] truncfile 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] usemem 2020-11-16T22:28:31.871 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] mmapcat 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] append_reader 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] append_writer 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dirperf 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] metaperf 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] devzero 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] feature 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] alloc 2020-11-16T22:28:31.872 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fault 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fstest 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_access_root 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] godown 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] resvtest 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] writemod 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] writev_on_pagefault 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] makeextents 2020-11-16T22:28:31.873 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] itrash 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] rename 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] multi_open_unlink 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c: In function 'run_child': 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:95:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.874 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~ 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:95:17: note: directive argument in the range [0, 2147483647] 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~ 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:95:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:166:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2020-11-16T22:28:31.875 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~ 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:166:17: note: directive argument in the range [0, 2147483647] 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~ 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr:fstest.c:166:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2020-11-16T22:28:31.876 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:31.877 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:31.877 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:31.879+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:31.932 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dmiperf 2020-11-16T22:28:31.966 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:31.980+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:31.970 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] unwritten_sync 2020-11-16T22:28:31.985 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] genhashnames 2020-11-16T22:28:32.014 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:28:32.015 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:32.023 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:32.026 INFO:tasks.workunit.client.0.smithi138.stderr:multi_open_unlink.c: In function 'main': 2020-11-16T22:28:32.027 INFO:tasks.workunit.client.0.smithi138.stderr:multi_open_unlink.c:101:4: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2020-11-16T22:28:32.027 INFO:tasks.workunit.client.0.smithi138.stderr: sts = attr_set(path, attrname, attrvalue, value_size, flags); 2020-11-16T22:28:32.027 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~ 2020-11-16T22:28:32.027 INFO:tasks.workunit.client.0.smithi138.stderr:In file included from multi_open_unlink.c:15: 2020-11-16T22:28:32.027 INFO:tasks.workunit.client.0.smithi138.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2020-11-16T22:28:32.028 INFO:tasks.workunit.client.0.smithi138.stderr: extern int attr_set (const char *__path, const char *__attrname, 2020-11-16T22:28:32.028 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~ 2020-11-16T22:28:32.066 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:32.081+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:32.095 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_holes 2020-11-16T22:28:32.166 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:32.181+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:32.182 INFO:tasks.workunit.client.0.smithi138.stderr:dmiperf.c: In function 'mkfile': 2020-11-16T22:28:32.182 INFO:tasks.workunit.client.0.smithi138.stderr:dmiperf.c:192:2: warning: 'attr_setf' is deprecated: Use fsetxattr instead [-Wdeprecated-declarations] 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr: if (attr_setf(fd, DMFATTRNAME, attr, DMFATTRLEN, ATTR_ROOT) < 0) { 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr: ^~ 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr:In file included from dmiperf.c:20: 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr:/usr/include/attr/attributes.h:143:12: note: declared here 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr: extern int attr_setf (int __fd, const char *__attrname, 2020-11-16T22:28:32.183 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~ 2020-11-16T22:28:32.188 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:32.188 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:32.188 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:32.188 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:32.188 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:32.197 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:32.270 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_writev 2020-11-16T22:28:32.277 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_truncate_cmtime 2020-11-16T22:28:32.279 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:32.294+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:32.291 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dirhash_collide 2020-11-16T22:28:32.330 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_rename_overwrite 2020-11-16T22:28:32.364 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:32.365 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.057235592000001, 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:32.366 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:32.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:32.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:32.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:32.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.056732704000002, 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.375 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:32.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:32.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:32.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:32.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:32.384 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 16.128061471999999, 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:32.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:32.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:32.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:32.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:32.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.394 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 15.410775000999999, 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:32.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:32.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:32.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.867937691, 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.405 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.166612788, 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:32.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:32.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:32.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:32.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.424 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.166081775, 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.425 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:32.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:32.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:32.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:32.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:32.434 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.434 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165960947, 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:32.435 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:32.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:32.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.444 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165878633, 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:32.445 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165786003999999, 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:32.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:32.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:32.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165703715999999, 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.465 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:32.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:32.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:32.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:32.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:32.475 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165621372, 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:32.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:32.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:32.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:32.485 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165546252, 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:32.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:32.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:32.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:32.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:32.495 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165467317999999, 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:32.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:32.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:32.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:32.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:32.504 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:32.505 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165389896000001, 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:32.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:32.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:32.515 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165309733999999, 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:32.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:32.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:32.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:32.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:32.524 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165227921, 2020-11-16T22:28:32.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:32.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:32.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.534 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165134535, 2020-11-16T22:28:32.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.544 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.165014139, 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:32.545 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:32.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:32.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.554 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 13.164914072, 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:32.555 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:32.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:32.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:32.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:32.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.564 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.565 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:32.565 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:32.565 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:32.566 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] holetest 2020-11-16T22:28:32.566 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_truncate_self 2020-11-16T22:28:32.566 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_dio 2020-11-16T22:28:32.567 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] af_unix 2020-11-16T22:28:32.567 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:32.580+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:32.567 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:32.667 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:32.681+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:32.705 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:32.705 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:32.705 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:32.705 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:32.714 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:32.729 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_stale_pmd 2020-11-16T22:28:32.793 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_cow_race 2020-11-16T22:28:32.797 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:32.811+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:32.802 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_fallocate 2020-11-16T22:28:32.804 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fsync-err 2020-11-16T22:28:32.869 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:32.883+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:32.880 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:28:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:28:32.881 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:32.889 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:32.967 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_write_ro 2020-11-16T22:28:32.970 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:32.984+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:33.023 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_ext4_dax_journal_corruption 2020-11-16T22:28:33.041 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_ext4_dax_inline_corruption 2020-11-16T22:28:33.058 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:33.059 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 315.56327058699998, 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:33.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:33.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 315.56315394400002, 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.071 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:33.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:33.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:33.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.081 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 315.56300557499998, 2020-11-16T22:28:33.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:33.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:33.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:33.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 315.56222214600001, 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:33.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:33.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:33.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:33.103 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 19.006185718000001, 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.104 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:33.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.113 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 18.293916228000001, 2020-11-16T22:28:33.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:28:33.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:28:33.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:28:33.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:28:33.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:28:33.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:28:33.123 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.817489533, 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:33.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:28:33.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:28:33.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:28:33.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:28:33.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.747267067999999, 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.133 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:33.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:28:33.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:28:33.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:28:33.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.143 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.097331337, 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:28:33.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:28:33.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:28:33.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.151 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.091584029, 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.152 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:33.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:28:33.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:28:33.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:28:33.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.161 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.090448025000001, 2020-11-16T22:28:33.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:28:33.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:28:33.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:28:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:28:33.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.089300929, 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.173 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:28:33.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:28:33.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:28:33.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.183 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.083359557000001, 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.184 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:28:33.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:28:33.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.081014818, 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.195 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:33.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:28:33.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:28:33.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:28:33.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:28:33.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:28:33.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:28:33.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.076268680999998, 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.208 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:33.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:28:33.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:28:33.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:28:33.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 15.557402138, 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:28:33.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:28:33.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 15.038385372, 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:28:33.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 13.861578203000001, 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.235 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:28:33.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 13.844886001000001, 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.243 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:28:33.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:28:33.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.815700424999999, 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.251 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:28:33.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:28:33.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:33.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:33.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:33.259 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:33.259 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:33.260 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_ofd_locks 2020-11-16T22:28:33.261 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_locks_execve 2020-11-16T22:28:33.261 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:28:33.261 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:33.262 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:33.274+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:33.265 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mmap_collision 2020-11-16T22:28:33.283 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] mmap-write-concurrent 2020-11-16T22:28:33.352 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] xfsctl 2020-11-16T22:28:33.361 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:33.375+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:33.389 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:33.389 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:33.389 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:28:33.389 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:33.398 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:33.460 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] bstat 2020-11-16T22:28:33.462 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:33.476+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:33.528 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_mtab 2020-11-16T22:28:33.540 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] getdevicesize 2020-11-16T22:28:33.563 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:33.577+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:33.565 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:33.565 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:33.575 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:33.630 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] preallo_rw_pattern_reader 2020-11-16T22:28:33.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~76s 2020-11-16T22:28:33.664 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:33.678+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:33.741 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:33.742 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 28.434018574, 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.743 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:33.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 23.433515686, 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.752 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:33.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:33.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.504844454000001, 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.770 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 16.787557982999999, 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:33.771 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:33.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:33.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:33.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 16.244720673, 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:33.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:33.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:33.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.54339577, 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:33.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:33.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:33.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.800 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542864757, 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:33.801 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:33.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:33.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:33.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.810 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542743929, 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.811 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:33.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:33.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:33.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:33.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542661615, 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.821 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:33.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:33.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:33.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:33.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542568985999999, 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.831 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:33.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:33.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542486697999999, 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.841 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:33.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:33.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542404354, 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.851 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542329234, 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542250299999999, 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542172878000001, 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:33.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:33.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:33.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.891 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.891 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:33.891 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542092716000001, 2020-11-16T22:28:33.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:33.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:33.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:33.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:33.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:33.901 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.542010903, 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:33.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:33.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:33.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:33.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:33.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:33.911 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.541917517, 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:33.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:33.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:33.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:33.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:33.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:33.921 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.541797121, 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:33.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:33.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:33.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:33.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:33.930 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:33.931 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.541697054, 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:33.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:33.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:33.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:33.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:33.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:33.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:33.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:33.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:33.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:34.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:34.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:34.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:34.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:34.144 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:34.145 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:34.145 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:34.145 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:34.145 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:34.146 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:34.147 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] preallo_rw_pattern_writer 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] ftrunc 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] trunc 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fs_perms 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] testx 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] looptest 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] locktest 2020-11-16T22:28:34.148 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] unwritten_mmap 2020-11-16T22:28:34.149 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.160+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.149 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:34.193 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] bulkstat_unlink_test 2020-11-16T22:28:34.247 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.261+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.280 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] bulkstat_unlink_test_modified 2020-11-16T22:28:34.288 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:34.288 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:34.289 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:34.289 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:34.298 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:34.342 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:34.348 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:34.362+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:34.367 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:34.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:34.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:34.390 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_dir_offset 2020-11-16T22:28:34.423 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:34.423 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:34.432 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:34.473 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:34.487+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:34.481 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_futimens 2020-11-16T22:28:34.498 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_immutable 2020-11-16T22:28:34.551 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.565+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.575 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] stale_handle 2020-11-16T22:28:34.598 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:34.598 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:34.607 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:34.621 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] pwrite_mmap_blocked 2020-11-16T22:28:34.651 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.666+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.691 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_dir_offset2 2020-11-16T22:28:34.734 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:34.734 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:34.743 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:34.785 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.799+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.834 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] seek_sanity_test 2020-11-16T22:28:34.852 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:34.867+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:34.861 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] seek_copy_test 2020-11-16T22:28:34.909 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:34.909 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:34.918 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:34.949 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_readdir_1 2020-11-16T22:28:34.953 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:34.967+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.054 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:35.069+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:35.082 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.082 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.091 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:35.113 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_readdir_2 2020-11-16T22:28:35.155 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:35.170+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:35.193 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fsync-tester 2020-11-16T22:28:35.217 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.218 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.228 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:35.262 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:28:35.271 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:35.285+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.312 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] nsexec 2020-11-16T22:28:35.344 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] cloner 2020-11-16T22:28:35.355 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.355 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.357 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:35.371+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.364 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:35.387 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] renameat2 2020-11-16T22:28:35.458 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:35.472+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.530 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.530 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.539 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:35.554 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_getcwd 2020-11-16T22:28:35.582 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:35.597+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:35.588 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] e4compact 2020-11-16T22:28:35.631 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] test-nextquota 2020-11-16T22:28:35.660 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:35.674+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:35.668 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.668 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.677 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:35.678 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] punch-alternating 2020-11-16T22:28:35.761 INFO:tasks.workunit.client.0.smithi138.stderr:t_getcwd.c: In function 'do_rename': 2020-11-16T22:28:35.761 INFO:tasks.workunit.client.0.smithi138.stderr:t_getcwd.c:40:2: warning: 'strncpy' specified bound 256 equals destination size [-Wstringop-truncation] 2020-11-16T22:28:35.761 INFO:tasks.workunit.client.0.smithi138.stderr: strncpy(c_name, prefix, BUF_SIZE); 2020-11-16T22:28:35.761 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:35.762 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:35.776+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.807 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] attr-list-by-handle-cursor-test 2020-11-16T22:28:35.843 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.843 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.853 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:35.889 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] listxattr 2020-11-16T22:28:35.893 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dio-interleaved 2020-11-16T22:28:35.896 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:35.911+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:35.935 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_dir_type 2020-11-16T22:28:35.963 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:35.978+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:35.979 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:35.979 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:35.988 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:36.064 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:36.078+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:36.077 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] dio-invalidate-cache 2020-11-16T22:28:36.113 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:36.113 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:28:36.123 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:36.132 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] stat_test 2020-11-16T22:28:36.137 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_encrypted_d_revalidate 2020-11-16T22:28:36.164 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:36.178+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:36.175 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] attr_replace_test 2020-11-16T22:28:36.248 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:36.249 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:36.257 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:36.301 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:36.315+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:36.365 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:36.379+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:36.372 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:36.381 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] swapon 2020-11-16T22:28:36.383 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:36.383 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:36.392 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] mkswap 2020-11-16T22:28:36.393 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:36.406 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_attr_corruption 2020-11-16T22:28:36.466 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:36.480+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:36.519 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:36.519 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:36.528 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:36.533 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_open_tmpfiles 2020-11-16T22:28:36.567 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:36.570 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:36.584+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:36.598 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:36.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:36.602 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:36.604 INFO:tasks.workunit.client.0.smithi138.stderr:swapon.c: In function 'main': 2020-11-16T22:28:36.604 INFO:tasks.workunit.client.0.smithi138.stderr:swapon.c:135:3: warning: 'p' may be used uninitialized in this function [-Wmaybe-uninitialized] 2020-11-16T22:28:36.604 INFO:tasks.workunit.client.0.smithi138.stderr: memcpy(p, buf, BUF_SIZE); 2020-11-16T22:28:36.604 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:36.644 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fscrypt-crypt-util 2020-11-16T22:28:36.651 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] bulkstat_null_ocount 2020-11-16T22:28:36.654 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] splice-test 2020-11-16T22:28:36.668 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:36.682+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:36.696 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:36.697 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:36.706 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:36.769 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:36.783+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:36.810 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fiemap-tester 2020-11-16T22:28:36.834 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:36.834 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:36.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:36.834 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:36.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.526337135999999, 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:36.835 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:36.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:36.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:36.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:36.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:36.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:36.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:36.844 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 26.525834247999999, 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:36.845 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:36.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:36.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:36.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:36.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:36.853 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 20.597163016, 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:36.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:36.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:36.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:36.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:37.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.065 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.879876544999998, 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:37.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:37.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:37.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:37.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:37.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:37.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.481 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.337039234999999, 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.635714331999999, 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:37.695 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:37.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:37.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:37.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:37.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.635183318999999, 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.111 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:38.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:38.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:38.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:38.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.635062490999999, 2020-11-16T22:28:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634980176999999, 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:38.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634887547999998, 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.546 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:38.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:38.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.63480526, 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:38.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634722916000001, 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.566 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:38.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:38.570 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:38.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634647795999999, 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.576 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634568861999998, 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.586 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:38.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:38.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634491440000001, 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.596 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:38.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:38.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:38.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634411278000002, 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634329465, 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634236079000001, 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634115683000001, 2020-11-16T22:28:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 17.634015615999999, 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:38.858 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:38.859 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:28:38.859 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] t_stripealign 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] open_by_handle 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] fssum 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout:Building log-writes 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] replay-log 2020-11-16T22:28:38.863 INFO:tasks.workunit.client.0.smithi138.stdout:Building perf 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout:Building aio-dio-regress 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-append-write-fallocate-race 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-fcntl-race 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-subblock-eof-read 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-append-write-read-race 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-invalidate-readahead 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-hole-filling-race 2020-11-16T22:28:38.864 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-last-ref-held-by-io 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-cycle-write 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-extend-stat 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aiocp 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-invalidate-failure 2020-11-16T22:28:38.865 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aiodio_sparse2 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-cow-race 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-write-verify 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout: [CC] aio-dio-eof-race 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout:Seed set to 1 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x366e7 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0xead53 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0xfc5c3 2020-11-16T22:28:38.866 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0xfca47 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr:aiodio_sparse2.c: In function 'main': 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr:aiodio_sparse2.c:404:2: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr: strncpy(filename, argv[argc-1], PATH_MAX); 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr:+ cd .. 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr:+ cp xfstests/ltp/fsx . 2020-11-16T22:28:38.867 INFO:tasks.workunit.client.0.smithi138.stderr:+ OPTIONS=-z 2020-11-16T22:28:38.868 INFO:tasks.workunit.client.0.smithi138.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2020-11-16T22:28:38.868 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2020-11-16T22:28:38.868 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2020-11-16T22:28:38.868 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support clone range, disabling! 2020-11-16T22:28:38.868 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support dedupe range, disabling! 2020-11-16T22:28:38.869 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:38.873+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:38.869 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:38.960 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:38.974+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.002 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:39.002 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:39.002 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:39.003 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:39.012 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:39.021 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:39.042 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:39.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:39.046 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:39.061 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:39.075+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:39.136 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0xff398 2020-11-16T22:28:39.144 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:39.144 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:39.154 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:39.195 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:39.209+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:39.262 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.276+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.326 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:39.327 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:39.336 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:39.379 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.394+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.464 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:39.478+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:39.472 INFO:tasks.workunit.client.0.smithi138.stdout:fallocating to largest ever: 0x100000 2020-11-16T22:28:39.482 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:39.482 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:39.493 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:39.565 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:39.580+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:39.666 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.680+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.667 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:39.667 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:39.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~82s 2020-11-16T22:28:39.677 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:39.766 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.781+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.865 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:39.865 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:39.867 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.881+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:39.876 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:39.968 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:39.983+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.013 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.013 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.023 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:40.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.083+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.124 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:28:40.155 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.156 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.165 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:40.170 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.271 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.285+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.297 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.297 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.306 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:40.372 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:40.385+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:40.450 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.451 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.461 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:40.504 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.519+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.574 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:40.587+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:40.603 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.603 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.614 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:40.674 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:40.688+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:40.776 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:40.790+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:40.791 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.792 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.802 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:40.876 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.890+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:40.930 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:40.930 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:40.940 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:40.977 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:40.991+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:41.047 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:41.074 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:41.074 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:41.077 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:41.091+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:41.085 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:41.179 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:41.192+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:41.197 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:41.218 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:41.218 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:41.219 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:41.219 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:41.219 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:41.221 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:41.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:41.259 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:41.259 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:41.260 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 323.76348750099999, 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:41.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:41.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:41.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:41.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:41.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:41.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:41.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:41.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:41.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:41.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 323.76337085799997, 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:41.272 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:41.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:41.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:41.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:41.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 323.76322248899999, 2020-11-16T22:28:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:41.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:41.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:41.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:42.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:42.107 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 323.76243906000002, 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:42.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:42.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:42.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:42.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:42.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:42.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.320 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 27.206402632, 2020-11-16T22:28:42.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.322 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:42.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:42.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:42.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:42.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:42.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:42.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:42.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.737 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.494133141999999, 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:42.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:42.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:42.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:42.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:42.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:42.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:28:42.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:42.943 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:28:42.944 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:42.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:28:42.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:42.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:42.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:43.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:28:43.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:28:43.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 25.017706446999998, 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:43.152 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:43.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:28:43.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:28:43.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:28:43.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.159 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:43.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.947483982000001, 2020-11-16T22:28:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:28:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:28:43.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:28:43.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:43.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:43.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:43.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:28:43.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:28:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:28:43.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:43.779 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.297548250999998, 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:43.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:28:43.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:28:43.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:28:43.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:43.787 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.291800942999998, 2020-11-16T22:28:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:28:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:28:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:28:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:28:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:28:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:28:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:44.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:28:44.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:44.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:28:44.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:28:44.406 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:44.406 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.290664938999999, 2020-11-16T22:28:44.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:28:44.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:28:44.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:28:44.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:44.614 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:28:44.614 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:28:44.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.614 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.614 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:28:44.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.616 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:28:44.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:44.618 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:28:44.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:44.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.289517842999999, 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:44.621 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.622 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:44.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:28:44.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:44.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:28:44.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:28:44.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:28:45.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:45.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:28:45.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:45.037 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.283576471, 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:28:45.038 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:28:45.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.242 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:45.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:45.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:28:45.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:28:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:28:45.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:45.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:28:45.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:28:45.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.281231731999998, 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:45.455 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:45.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:28:45.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:45.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:28:45.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:28:45.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:45.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:45.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:45.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:28:45.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:28:45.874 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.874 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 24.276485595, 2020-11-16T22:28:45.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.876 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:45.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:45.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:45.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:45.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:28:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:28:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:28:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:28:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:28:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.757619051999999, 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:28:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:28:46.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:28:46.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:28:46.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:46.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:28:46.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:28:46.299 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.299 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:46.502 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.502 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:28:46.502 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 23.238602285999999, 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:28:46.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:28:46.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:46.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:28:46.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:28:46.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.713 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.061795116999999, 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:28:46.714 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:46.715 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:46.715 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:46.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:46.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:46.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:46.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:46.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:28:46.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:46.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:28:46.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:46.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.045102915000001, 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:46.926 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:46.928 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:28:47.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:28:47.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:28:47.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:28:47.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:47.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:28:47.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:28:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.015917339000001, 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:28:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:28:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:47.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:47.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:28:47.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:47.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:28:47.549 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:47.550 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:47.550 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:47.550 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:47.550 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:47.550 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:47.551 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:28:47.552 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:47.553 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:47.554 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:47.565+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:47.652 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:47.667+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:47.696 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:9467 4.3 4:cd35a60f:::10000001887.00000000:head [zero 3619~1717] snapc 1=[] ondisk+write+known_if_redirected e93)", 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:47.709003+0000", 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0025347880000000001, 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0025538869999999999, 2020-11-16T22:28:47.697 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 9467 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.698 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709003+0000", 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:47.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709003+0000", 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1019e-05 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709024+0000", 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8890000000000004e-06 2020-11-16T22:28:47.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:47.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709028+0000", 2020-11-16T22:28:47.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.81e-06 2020-11-16T22:28:47.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709035+0000", 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.039e-06 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709038+0000", 2020-11-16T22:28:47.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00086728100000000002 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709905+0000", 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1131000000000002e-05 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.709957+0000", 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00095316900000000002 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:4898 3.0 3:02ac1934:::200.00000009:head [write 1960122~1921 [fadvise_dontneed] in=1921b] snapc 0=[] ondisk+write+known_if_redirected+full_force e93)", 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:47.711160+0000", 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00037834199999999999, 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00045560899999999999, 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:47.906 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711160+0000", 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:47.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711160+0000", 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4192000000000001e-05 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711194+0000", 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711199+0000", 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8939999999999999e-06 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:47.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711208+0000", 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711212+0000", 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3862999999999997e-05 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711246+0000", 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9307e-05 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:47.711305+0000", 2020-11-16T22:28:47.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014561500000000001 2020-11-16T22:28:48.114 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:48.114 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:48.114 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:48.114 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:48.115 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:28:48.115 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:28:48.115 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:48.117 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:48.117 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:48.117 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:48.117 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:48.117 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:48.118 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:48.118 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:48.118 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:48.119 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:48.119 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.130+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.216 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:48.231+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:48.296 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:48.297 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:48.307 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:48.317 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.331+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.418 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.432+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.504 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:48.504 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:48.516 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:48.518 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.532+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.619 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.633+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.653 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:48.654 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:48.663 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:48.720 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:48.734+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:48.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~91s 2020-11-16T22:28:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:28:48.821 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.835+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:48.828 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:48.828 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:48.838 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:48.922 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:48.936+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.002 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.002 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.011 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:49.052 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.066+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.123 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:49.137+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:49.151 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.151 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.161 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:49.223 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.237+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.324 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:49.338+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:49.341 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.341 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.351 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:49.434 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:49.449+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:49.525 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.539+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.541 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:49.626 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.640+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.668 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.668 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.677 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:49.760 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.774+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:49.828 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:49.842+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:49.841 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:49.841 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:49.850 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:49.933 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:49.947+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:50.008 INFO:tasks.workunit.client.0.smithi138.stdout:10000 trunc from 0xbf6ab to 0x628a6 2020-11-16T22:28:50.014 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:50.014 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:50.023 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:50.064 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:50.078+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:50.119 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:50.130 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:50.144+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:50.168 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:50.168 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:50.178 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:50.231 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:50.245+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:50.265 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:50.286 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:50.286 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:50.287 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:50.287 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:50.287 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:50.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:50.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:50.332 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:50.346+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:50.378 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:50.379 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:50.391 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:50.432 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:50.446+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:50.533 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:50.547+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.589 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 45.281314709999997, 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:50.590 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:50.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:50.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:50.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:50.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.280811821999997, 2020-11-16T22:28:50.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.600 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:50.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:50.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:50.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.608 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 34.352140589999998, 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:50.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:50.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:50.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:50.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.618 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.634854119000003, 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:50.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:50.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:50.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:50.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.628 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.092016809, 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:50.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:50.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:50.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:50.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.638 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.390691906000001, 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:50.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:50.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:50.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.648 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.390160893000001, 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:50.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:50.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:50.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.658 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.390040065000001, 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.668 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389957751000001, 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:50.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:50.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:50.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:50.678 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389865122, 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:50.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:50.688 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389782833999998, 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:50.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:50.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:50.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:50.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:50.697 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:50.698 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389700489999999, 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:50.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:50.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:50.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:50.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.707 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389625370000001, 2020-11-16T22:28:50.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:50.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:50.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:50.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:50.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.718 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389546436, 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:50.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:50.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:50.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.728 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389469013999999, 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:50.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:50.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.738 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389388852, 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:50.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:50.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:50.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:50.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.748 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389307038999998, 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:50.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:50.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:50.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:50.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389213652999999, 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:50.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:50.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.768 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.389093256999999, 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:50.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:50.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:50.778 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.388993190000001, 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:50.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:50.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:50.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:50.787 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.788 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:50.788 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:50.788 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:50.789 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:28:50.790 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:50.803+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:50.791 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:50.890 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:50.904+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:50.922 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:50.922 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:50.922 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:50.923 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:50.931 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:50.990 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:51.004+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:51.056 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:51.056 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:51.056 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:51.065 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:51.091 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:51.105+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:51.191 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:51.192 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 333.696453846, 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:51.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:51.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:51.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:51.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 333.69633720299998, 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.204 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:51.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:51.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:51.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:51.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:51.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 333.696188834, 2020-11-16T22:28:51.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:51.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:51.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:51.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:51.226 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 333.69540540499997, 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:51.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:51.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:51.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:51.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:51.236 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 37.139368976999997, 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:51.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:51.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:51.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:51.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:51.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 36.427099487, 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:28:51.247 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:28:51.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:28:51.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:28:51.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.254 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.950672791999999, 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.255 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:28:51.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:28:51.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:28:51.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.880450326999998, 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.263 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:28:51.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:28:51.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:28:51.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:28:51.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.273 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.230514595999999, 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:28:51.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:28:51.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:28:51.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.281 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.224767288000002, 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.282 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:51.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:28:51.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:28:51.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:28:51.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:28:51.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.223631284, 2020-11-16T22:28:51.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:28:51.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:28:51.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:28:51.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:28:51.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:28:51.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.222484188000003, 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.303 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:51.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:28:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:28:51.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:28:51.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.313 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.216542816, 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:51.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:28:51.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:28:51.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:28:51.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:28:51.323 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.214198076999999, 2020-11-16T22:28:51.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:28:51.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:28:51.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:28:51.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:28:51.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.209451940000001, 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.335 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:28:51.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:28:51.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:28:51.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:51.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:28:51.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.345 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.690585397, 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:51.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:28:51.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:28:51.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.353 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.171568631, 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:51.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:28:51.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:28:51.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.361 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.994761462, 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:28:51.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:28:51.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:28:51.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:28:51.369 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 31.978069260000002, 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:28:51.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:28:51.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:28:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:28:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 30.948883683999998, 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:28:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:28:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:51.385 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:51.387 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:51.388 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:51.401+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:51.488 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:51.502+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:51.534 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:51.535 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:51.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:5880 3.0 3:02ac1934:::200.00000009:head [write 3483538~1921 [fadvise_dontneed] in=1921b] snapc 0=[] ondisk+write+known_if_redirected+full_force e94)", 2020-11-16T22:28:51.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:51.547714+0000", 2020-11-16T22:28:51.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0017232580000000001, 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001742078, 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547714+0000", 2020-11-16T22:28:51.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547714+0000", 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3426999999999997e-05 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547747+0000", 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5759999999999996e-06 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:51.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547753+0000", 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0268000000000001e-05 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547763+0000", 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.451e-06 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547768+0000", 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1888000000000003e-05 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:51.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547810+0000", 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5896000000000005e-05 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.547906+0000", 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000192506 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:10886 4.3 4:cd35a60f:::10000001887.00000000:head [read 626688~0 [654@453283]] snapc 0=[] ondisk+read+known_if_redirected e94)", 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:51.549195+0000", 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00024143, 2020-11-16T22:28:51.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00030513699999999998, 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 10886 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.543 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549195+0000", 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549195+0000", 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8674999999999999e-05 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549224+0000", 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9049999999999999e-06 2020-11-16T22:28:51.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549228+0000", 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8140000000000003e-06 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549235+0000", 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2799999999999999e-06 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549238+0000", 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0664000000000002e-05 2020-11-16T22:28:51.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549279+0000", 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5699000000000006e-05 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:51.549374+0000", 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000179037 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:28:51.549 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:28:51.550 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:51.550 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:51.589 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:51.603+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:51.683 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:51.683 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:51.684 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:51.684 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:51.684 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:51.689 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:51.703+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:51.693 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:51.790 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:51.804+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:51.840 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:51.840 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:51.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:51.840 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:51.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.532362554999999, 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:51.841 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:51.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:51.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:51.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:51.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:51.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:56.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.531859666999999, 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.025 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:56.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:56.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:56.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:56.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:56.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.236 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.603188435, 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:28:56.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:28:56.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:28:56.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:28:56.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:28:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:28:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:28:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 34.885901963999999, 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:28:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:28:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:28:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:28:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 34.343064654000003, 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:28:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:28:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.641739751000003, 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:28:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:28:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.641208738000003, 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:28:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:28:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.641087910000003, 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:28:56.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:28:56.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:28:56.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.641005595999999, 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:28:56.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:28:56.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:28:56.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.509 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640912966999998, 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:28:56.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:28:56.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:28:56.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:28:56.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640830678999997, 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:28:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:28:56.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:28:56.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:28:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640748334999998, 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:28:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:28:56.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:28:56.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:28:56.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.539 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640673215, 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:28:56.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:28:56.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:28:56.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:28:56.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.549 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640594280999998, 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:28:56.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:28:56.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:28:56.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:28:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640516859000002, 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:28:56.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:28:56.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:28:56.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:56.569 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640436696999998, 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.570 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:28:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:28:56.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:28:56.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:28:56.578 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:56.579 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640354883999997, 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:56.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:28:56.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:28:56.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:28:56.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.588 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640261498000001, 2020-11-16T22:28:56.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:28:56.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.593 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:28:56.594 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.598 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640141102000001, 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:28:56.599 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:28:56.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:28:56.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:28:56.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.608 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.640041035000003, 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:28:56.609 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:28:56.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:28:56.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:28:56.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:28:56.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:56.619 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:56.619 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:56.619 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:28:56.621 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:28:56.622 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:56.634+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:56.622 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:56.721 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:56.735+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:56.755 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:56.755 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:56.755 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:28:56.755 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:56.765 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:56.808 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:56.821 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:56.835+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:56.829 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:56.830 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:28:56.830 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:28:56.830 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:56.830 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:56.832 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:56.832 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:56.894 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:56.894 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:56.903 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:56.945 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:56.959+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.023 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:57.037+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:57.079 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.080 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.090 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:57.124 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.138+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.225 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:57.239+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:57.247 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.247 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.259 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:57.326 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.340+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2020-11-16T22:28:57.427 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:57.441+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:57.432 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.432 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.442 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:57.528 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.542+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.628 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.642+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.629 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.629 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.641 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:57.729 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.743+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.774 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.775 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.785 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:57.868 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:57.882+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:57.931 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:57.945+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:57.959 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:57.959 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:57.968 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:58.032 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.046+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.133 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.147+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.137 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:58.137 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:58.147 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:58.234 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.248+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.311 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:58.312 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:58.324 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:58.335 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:58.349+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:58.436 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:58.450+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:58.503 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:58.503 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:58.513 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:58.557 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.571+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.622 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:28:58.637 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.652+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.661 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:58.661 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:58.672 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:28:58.738 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:58.752+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:58.804 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:28:58.805 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:28:58.815 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:28:58.834 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:28:58.838 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.853+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.939 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:58.954+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:58.983 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:28:58.985 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:58.985 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:28:58.985 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:28:58.985 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:28:58.985 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:58.995 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:28:59.004 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:28:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:28:59.005 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:28:59.005 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:28:59.005 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:28:59.007 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:28:59.007 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:28:59.040 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:59.055+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:59.126 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:59.127 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 341.63125722500001, 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:28:59.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:28:59.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:28:59.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:28:59.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:28:59.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 341.631140582, 2020-11-16T22:28:59.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:59.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:28:59.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 341.63099221300001, 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.151 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:28:59.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:28:59.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:28:59.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:28:59.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:28:59.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:59.162 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 341.63020878399999, 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:28:59.172 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.074172355999998, 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.173 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:59.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:28:59.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:28:59.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:28:59.183 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 44.361902866000001, 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:28:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.885476171000001, 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:28:59.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:28:59.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.815253706, 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.200 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:28:59.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:28:59.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:28:59.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:28:59.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:59.210 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.165317975000001, 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:59.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:28:59.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:28:59.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:28:59.217 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:59.218 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.159570666999997, 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.219 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:28:59.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:28:59.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:28:59.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.158434663000001, 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:28:59.229 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.230 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:28:59.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:28:59.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:28:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:28:59.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:28:59.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.157287566999997, 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.242 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:28:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:28:59.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:28:59.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:28:59.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.151346195000002, 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.253 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:28:59.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:28:59.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:28:59.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:28:59.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.149001456000001, 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.264 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:59.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:28:59.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:28:59.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:28:59.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:28:59.273 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:59.274 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.144255319000003, 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:28:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:28:59.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:28:59.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:28:59.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:28:59.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:28:59.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:59.285 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 41.625388776000001, 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:28:59.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:28:59.293 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 41.106372010000001, 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:28:59.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:28:59.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:28:59.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:28:59.301 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 39.929564841000001, 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:28:59.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:28:59.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:28:59.309 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 39.912872639, 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:28:59.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:28:59.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:28:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 38.883687063000004, 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:28:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:28:59.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:28:59.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:28:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:28:59.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:28:59.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.325 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:28:59.326 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:28:59.326 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:28:59.332 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:59.346+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:59.332 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:28:59.433 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:59.447+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:59.467 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:13502 4.3 4:cd35a60f:::10000001887.00000000:head [zero 491338~11154] snapc 1=[] ondisk+write+known_if_redirected e95)", 2020-11-16T22:28:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:59.481122+0000", 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00069206699999999999, 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00071646100000000005, 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 13502 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481122+0000", 2020-11-16T22:28:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481122+0000", 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1413999999999999e-05 2020-11-16T22:28:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481143+0000", 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3299999999999998e-06 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481149+0000", 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8870000000000004e-06 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481158+0000", 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3870000000000003e-06 2020-11-16T22:28:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481164+0000", 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9765e-05 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481194+0000", 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3423999999999994e-05 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:59.481257+0000", 2020-11-16T22:28:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013520700000000001 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:28:59.475 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:59.477 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:28:59.534 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:59.548+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:59.616 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:28:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:28:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:28:59.617 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:28:59.627 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:28:59.635 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:28:59.649+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:28:59.735 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:28:59.750+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:28:59.779 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:28:59.779 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 54.471843497000002, 2020-11-16T22:28:59.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:28:59.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:59.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:59.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:28:59.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:28:59.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:59.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:28:59.789 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:59.790 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 49.471340609000002, 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:28:59.791 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:28:59.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:28:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:28:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:28:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:28:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:28:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:28:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 43.542669377000003, 2020-11-16T22:28:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:00.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:00.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:00.002 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:00.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:00.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:00.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:00.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:00.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:00.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:00.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:00.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:00.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:00.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.215 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:00.418 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:00.418 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 42.825382906000002, 2020-11-16T22:29:00.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:00.418 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:00.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:00.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:00.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:00.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:00.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:00.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:00.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:00.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:00.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:00.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:00.834 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:00.834 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:00.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:145 3.2 3:5e41ab69:::200.00000005:head [write 0~2864040 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 42.282545595999999, 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999, 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:00.835 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.509874+0000", 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037697239999999999 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:00.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513644+0000", 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4479999999999999e-06 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513651+0000", 2020-11-16T22:29:00.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1637000000000001e-05 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513662+0000", 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5149999999999997e-06 2020-11-16T22:29:00.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:01.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513668+0000", 2020-11-16T22:29:01.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7954000000000001e-05 2020-11-16T22:29:01.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:01.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513726+0000", 2020-11-16T22:29:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010560699999999999 2020-11-16T22:29:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513831+0000", 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010260900000000001 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.513934+0000", 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.017318118 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.531252+0000", 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031107589999999998 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534363+0000", 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7077999999999998e-05 2020-11-16T22:29:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534420+0000", 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6489999999999994e-05 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:01.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.534496+0000", 2020-11-16T22:29:01.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024621938999999999 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.563 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.581220692999999, 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:01.564 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:01.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:01.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:01.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:01.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580689679999999, 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:01.980 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:29:01.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:29:01.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:01.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:29:02.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:29:02.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:29:02.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:29:02.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:29:02.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:29:02.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:29:02.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.396 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580568851999999, 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:02.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:02.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:02.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:02.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:02.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:02.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580486538000002, 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:02.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:02.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:02.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580393909000001, 2020-11-16T22:29:02.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:02.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:02.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:02.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:02.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580311621, 2020-11-16T22:29:02.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:02.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:02.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:02.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:02.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.842 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580229277000001, 2020-11-16T22:29:02.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:02.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:02.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.852 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580154157000003, 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:02.853 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:02.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:02.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:02.862 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.580075223000001, 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:02.863 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:02.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:02.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579997800999998, 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579917639000001, 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579835826, 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579742439999997, 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579622043999997, 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:03.330 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 40.579521976999999, 2020-11-16T22:29:03.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:03.330 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:03.330 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:03.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:03.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:03.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:03.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.339 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.340 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.340 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:03.340 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:03.340 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:03.341 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:03.342 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:29:03.343 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.355+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.343 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:03.442 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.457+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.473 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:03.473 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:03.473 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:03.474 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:03.483 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:03.528 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:03.543 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.557+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.548 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:03.548 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:03.548 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:03.549 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:03.549 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:03.551 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:03.551 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:03.615 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:03.615 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:03.624 INFO:tasks.workunit.client.0.smithi138.stdout:20000 mapwrite 0xe10fb thru 0xe4c85 (0x3b8b bytes) 2020-11-16T22:29:03.625 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:03.644 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.658+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.744 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.759+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.756 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:03.756 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:03.766 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:03.846 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:03.860+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:03.917 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:03.918 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:03.929 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:03.947 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:03.961+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.047 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.061+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.111 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.112 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.122 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:04.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~107s 2020-11-16T22:29:04.149 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.162+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.249 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:04.263+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:04.271 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.272 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.281 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:04.350 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:04.365+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:04.414 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.414 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.424 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:04.467 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.481+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.552 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.566+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.554 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.554 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.564 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:04.653 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.667+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.736 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.737 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.747 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:04.788 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.802+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.856 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.869+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:04.902 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:04.902 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:04.914 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:04.956 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:04.970+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.057 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:05.071+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:05.061 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:05.061 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:05.071 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:05.157 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:05.171+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.250 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:05.250 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:05.257 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:05.272+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:05.262 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:05.345 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:29:05.359 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:05.374+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:05.392 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:05.393 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:05.402 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:05.460 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:05.474+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:05.530 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:05.530 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:05.539 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:05.552 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:05.583 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:05.597+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.661 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:05.675+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.667 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:05.667 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:05.677 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:05.698 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:05.719 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:05.720 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:05.720 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:05.720 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:05.720 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:05.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:05.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:05.763 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:05.777+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.822 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:05.822 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:05.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.822 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:8903 4.3 4:cd35a60f:::10000001887.00000000:head [trimtrunc 1254@807890] snapc 1=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:05.833176+0000", 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.003845663, 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038638100000000001, 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:05.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833176+0000", 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833176+0000", 2020-11-16T22:29:05.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9203000000000003e-05 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833225+0000", 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3520000000000007e-06 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833234+0000", 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5277999999999998e-05 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833249+0000", 2020-11-16T22:29:05.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0800000000000002e-06 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833255+0000", 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1671e-05 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833317+0000", 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109954 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:05.833427+0000", 2020-11-16T22:29:05.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025053800000000002 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:05.829 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:05.832 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:05.863 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:05.877+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:05.963 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:05.977+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:05.975 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:05.975 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:29:05.975 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:29:05.975 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:29:05.976 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:05.985 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:29:05.986 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:06.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:06.083+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:06.166 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:06.180+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:06.176 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:06.177 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 60.868508841999997, 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:06.178 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:06.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:06.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:06.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:06.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:06.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 55.868005953999997, 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:06.187 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:06.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:06.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:06.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:06.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:06.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:06.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:06.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:06.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.399 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 49.939334721999998, 2020-11-16T22:29:06.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:06.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:06.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:06.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:06.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:06.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:06.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 49.222048250999997, 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:06.613 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:06.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:06.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:06.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:06.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:06.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:06.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:06.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:06.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:06.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:06.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:06.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:06.825 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.977886038000001, 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:06.826 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:06.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:06.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:07.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:07.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:07.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:07.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:07.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:07.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:07.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:07.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:07.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:07.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.038 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.977355025000001, 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:07.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:07.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:07.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:07.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:29:07.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:07.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:29:07.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:07.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:29:07.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:29:07.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:07.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:07.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.453 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.977234197000001, 2020-11-16T22:29:07.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:07.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:07.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:07.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:07.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:07.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.977151882999998, 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:07.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:07.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:07.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:07.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.977059253999997, 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:08.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:08.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:08.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:08.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:08.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.295 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976976966000002, 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:08.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:08.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:08.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:08.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:08.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976894622000003, 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.508 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:08.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:08.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:08.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:08.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.517 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976819501999998, 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:08.518 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:08.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:08.528 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976740567999997, 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:08.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:08.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:08.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:08.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976663146, 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:08.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:08.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:08.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:08.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976582983999997, 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.549 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:08.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:08.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:08.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:08.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.559 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976501171000002, 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:08.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:08.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:08.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976407784999999, 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.772 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:08.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:08.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:08.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:08.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976287388999999, 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:08.782 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:08.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:08.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:08.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:08.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:08.792 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.976187322000001, 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:08.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:08.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:08.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:08.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.802 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 5.5759143379999996, 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:08.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:08.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:08.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:08.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:08.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:08.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:08.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:08.813 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:08.813 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:08.814 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:29:08.815 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:08.828+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:08.816 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:08.914 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:08.928+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:08.951 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:08.951 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:29:08.951 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:9541 3.2 e97/82 3:4272d415:::200.0000000b:head v 97'398, mlcod=97'396)", 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:29:08.965828+0000", 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00017426400000000001, 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019004699999999999, 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:29:08.952 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965828+0000", 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.953 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:08.954 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965828+0000", 2020-11-16T22:29:08.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4106999999999997e-05 2020-11-16T22:29:08.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.954 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.954 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965862+0000", 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965867+0000", 2020-11-16T22:29:08.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6350000000000006e-06 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965875+0000", 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9940000000000002e-06 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965879+0000", 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4251e-05 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965923+0000", 2020-11-16T22:29:08.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2529000000000001e-05 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:29:08.965946+0000", 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118113 2020-11-16T22:29:08.958 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2020-11-16T22:29:08.959 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:08.961 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:08.966 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:08.988 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:08.993 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:08.993 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:09.015 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:09.029+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:09.096 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.097 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.106 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:09.150 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.164+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.216 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.230+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.237 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.237 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.247 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:09.317 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:09.331+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:09.379 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.379 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.389 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:09.418 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.433+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.520 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.534+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.566 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.566 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.576 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:09.620 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.634+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.720 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.720 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.722 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:09.736+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:09.730 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:09.822 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:09.836+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:09.866 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:09.866 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:09.876 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:09.923 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:09.937+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:09.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~113s 2020-11-16T22:29:09.994 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:29:10.006 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.006 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.016 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:10.024 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:10.037+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:10.124 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.139+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.148 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.148 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.157 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:10.225 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.240+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.308 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.309 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.320 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:10.326 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.340+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.427 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:10.440+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:10.460 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.460 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.470 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:10.528 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.542+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.629 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.643+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.659 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.659 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.669 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:10.729 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:10.742+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:10.803 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.803 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.814 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:10.857 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.871+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.931 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:10.945+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:10.946 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:10.946 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:10.955 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:10.994 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:11.032 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:11.045+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:11.127 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:11.127 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:11.132 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:11.146+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:11.138 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:11.141 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:11.162 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:11.163 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:11.163 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:11.163 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:11.163 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:11.165 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:11.166 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:11.233 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:11.246+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:11.274 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:11.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:9991 3.2 3:4272d415:::200.0000000b:head [write 1439108~1922 [fadvise_dontneed] in=1922b] snapc 0=[] ondisk+write+known_if_redirected+full_force e98)", 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:11.284990+0000", 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0043800100000000002, 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044052429999999997, 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:29:11.275 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.284990+0000", 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.284990+0000", 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6903999999999993e-05 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285077+0000", 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5580000000000001e-06 2020-11-16T22:29:11.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285085+0000", 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4516e-05 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285100+0000", 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9209999999999997e-06 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285106+0000", 2020-11-16T22:29:11.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016233799999999999 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285268+0000", 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8276000000000002e-05 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285326+0000", 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6049000000000001e-05 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.285422+0000", 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002156898 2020-11-16T22:29:11.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:11.287579+0000", 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00258946 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:11.282 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:11.283 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:11.283 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:11.283 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:11.283 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:11.284 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:11.368 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:11.382+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:11.434 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:11.448+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:11.462 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:11.463 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:29:11.463 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:29:11.463 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:29:11.463 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:11.473 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:11.535 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:11.550+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:11.636 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:11.651+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:11.658 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 66.350411429000005, 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.349908540999998, 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:11.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:11.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:11.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:11.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:11.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:12.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:12.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:12.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 55.421237308999999, 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:12.085 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:12.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:12.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:12.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:12.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:12.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:12.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:12.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:12.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:12.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:12.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:12.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:12.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 54.703950837999997, 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:12.502 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:12.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:12.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:12.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:12.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:12.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:12.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:12.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:12.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:12.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:12.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:12.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:12.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:12.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:12.917 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.459788625000002, 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:12.918 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:12.919 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:12.919 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:12.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:13.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:13.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:13.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:13.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:13.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:13.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:13.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:13.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:13.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:13.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:13.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:13.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.335 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:13.335 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:13.335 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.459257612000002, 2020-11-16T22:29:13.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:29:13.538 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:13.538 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:13.538 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:13.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:13.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:29:13.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:29:13.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:29:13.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:13.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:29:13.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:29:13.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:29:13.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:13.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:29:13.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:13.751 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:13.751 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:13.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.751 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.459136784000002, 2020-11-16T22:29:13.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:13.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:13.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:13.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:13.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:14.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:14.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:14.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:14.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:14.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:14.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.459054469999998, 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.371 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:14.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:14.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:14.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.381 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458961840999997, 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:14.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:14.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:14.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:14.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458879553000003, 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:14.392 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:14.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:14.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:14.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.402 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458797208999997, 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:14.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:14.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:14.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.412 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458722088999998, 2020-11-16T22:29:14.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:14.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:14.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458643154999997, 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.424 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:14.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:14.433 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:14.434 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458565733, 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:14.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:14.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:14.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:14.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458485570999997, 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.647 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:14.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:14.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:14.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:14.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458403758000003, 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.657 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:14.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:14.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:14.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:14.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458310372, 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.667 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:14.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:14.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:14.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458189976, 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:14.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:14.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:14.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.458089909000002, 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:14.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:14.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:14.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:14.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.696 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.057816924999999, 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:14.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:14.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:14.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:14.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:14.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:14.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:14.706 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:14.707 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:29:14.708 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:14.710 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:14.722+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:14.710 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:14.808 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:14.821+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:14.847 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:14.847 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:14.847 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:14.848 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:14.857 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:14.865 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:14.886 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:14.887 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:14.887 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:14.887 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:14.887 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:14.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:14.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:14.909 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:14.923+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:14.991 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:14.992 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.001 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:15.009 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:15.022+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:15.110 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.124+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.169 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.170 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.180 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:15.221 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.235+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.312 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:15.325+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:15.320 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.320 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.330 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:15.413 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.427+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.503 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.503 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.514 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:15.514 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.528+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.614 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:15.627+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:15.649 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.650 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.660 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:15.715 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.729+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~118s 2020-11-16T22:29:15.791 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.792 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.801 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:15.842 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.856+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.916 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:15.930+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:15.970 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:15.970 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:15.980 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:16.012 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:29:16.017 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.030+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.117 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.130+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.153 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.153 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.163 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:16.218 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:16.232+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:16.307 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.307 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.317 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:16.318 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.332+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.420 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.433+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.459 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.459 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.470 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:16.553 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.568+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.621 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:16.635+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:16.663 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.663 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.675 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:16.721 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:16.735+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:16.806 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.806 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.815 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:16.822 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:16.835+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:16.891 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:16.923 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:16.937+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:16.984 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:16.984 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:16.994 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:17.024 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:17.037+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:17.036 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:17.056 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:17.056 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:17.056 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:17.057 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:17.057 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:17.059 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:17.059 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:17.125 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:17.139+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.164 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 359.66878494399998, 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:29:17.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:29:17.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:17.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:29:17.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:17.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.175 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 359.66866830100003, 2020-11-16T22:29:17.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:29:17.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:29:17.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:17.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:29:17.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:17.187 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 359.66851993199998, 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:29:17.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:29:17.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:29:17.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 359.66773650300001, 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:29:17.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:29:17.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:29:17.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.208 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.111700075000002, 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:17.209 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:17.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:29:17.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:29:17.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:29:17.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:29:17.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.399430584999998, 2020-11-16T22:29:17.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:17.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:17.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:29:17.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:29:17.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:29:17.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:29:17.631 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.923003889999997, 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.632 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:17.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:29:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:29:17.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:17.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:29:17.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:17.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.842 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.852781425000003, 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:17.843 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:17.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:29:17.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:17.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:17.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:29:18.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:29:18.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:29:18.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:29:18.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:18.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:29:18.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:29:18.258 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.258 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.202845693999997, 2020-11-16T22:29:18.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:18.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:29:18.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:29:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:29:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.197098386, 2020-11-16T22:29:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:29:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:18.470 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:18.470 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:18.470 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:18.470 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:29:18.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:29:18.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:29:18.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:29:18.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:18.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:18.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:29:18.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.683 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.195962381999998, 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:18.684 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:18.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:29:18.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:18.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:29:18.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:29:18.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:29:18.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:29:18.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:18.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:18.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:19.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:29:19.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:29:19.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.194815286000001, 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:19.100 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:19.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:29:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:29:19.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:19.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:29:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:29:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:29:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:19.313 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.188873913999998, 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:29:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:19.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:19.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:29:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:29:19.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:29:19.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:29:19.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.526 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.186529174999997, 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:29:19.527 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:19.528 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:19.730 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:19.730 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:19.730 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:19.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:29:19.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:19.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:29:19.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:29:19.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.938 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:19.938 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:29:19.938 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.939 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:29:19.940 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.941 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:19.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:19.943 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 60.181783037999999, 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:19.944 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:19.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:19.946 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:29:19.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:19.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:19.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:20.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:29:20.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:29:20.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:20.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:29:20.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:29:20.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:20.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:29:20.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:29:20.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:20.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:29:20.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:29:20.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:20.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:29:20.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:29:20.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:20.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:29:20.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:29:20.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:20.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:29:20.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:29:20.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.662916494999997, 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:20.364 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:29:20.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:29:20.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:20.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:29:20.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:29:20.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:29:20.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.373 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:20.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:29:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.143899728999997, 2020-11-16T22:29:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:29:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:20.574 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:20.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:20.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:29:20.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:20.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:29:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:29:20.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:20.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:29:20.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:29:20.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:29:20.784 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 57.967092559999998, 2020-11-16T22:29:20.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:20.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:29:20.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:29:20.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:20.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:29:20.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:29:20.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.995 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 57.950400358000003, 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:20.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:20.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:20.998 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:20.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:21.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:29:21.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:29:21.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:21.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:29:21.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:21.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:29:21.206 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:21.207 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.921214782, 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:21.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:21.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:29:21.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:29:21.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:21.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:29:21.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:29:21.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:21.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:29:21.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:21.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:21.417 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:21.418 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:29:21.419 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:21.420 INFO:tasks.workunit.client.0.smithi138.stdout:30000 mapwrite 0xb7634 thru 0xbbad6 (0x44a3 bytes) 2020-11-16T22:29:21.421 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:21.422 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:21.433+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:21.520 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:21.535+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:21.570 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:21.577 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:21.577 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:11805 3.0 3:07a02702:::200.0000000c:head [write 62355~1922 [fadvise_dontneed] in=1922b] snapc 0=[] ondisk+write+known_if_redirected+full_force e99)", 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:21.590423+0000", 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001945081, 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019642650000000002, 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:21.578 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590423+0000", 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590423+0000", 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2727999999999999e-05 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590495+0000", 2020-11-16T22:29:21.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0159999999999997e-06 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590504+0000", 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5739999999999998e-05 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.581 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:21.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590520+0000", 2020-11-16T22:29:21.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6409999999999996e-06 2020-11-16T22:29:21.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:21.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590527+0000", 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7356e-05 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590554+0000", 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2958000000000003e-05 2020-11-16T22:29:21.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:21.590617+0000", 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000194439 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:21.788 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:21.789 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:21.789 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:21.791 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:21.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:21.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:21.792 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:21.793 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:21.804+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:21.891 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:21.905+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:21.968 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:21.968 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:21.978 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:22.022 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:22.036+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:22.031 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:29:22.092 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.107+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.123 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.124 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.134 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:22.192 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.206+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.269 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.269 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.278 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:22.319 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.333+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.393 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:22.408+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:22.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2020-11-16T22:29:22.442 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.443 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.452 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:22.495 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:22.509+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:22.575 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.575 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.584 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:22.596 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.610+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.696 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.711+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.736 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.737 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.748 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:22.797 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:22.811+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:22.898 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:22.912+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:22.914 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:22.998 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:23.013+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:23.091 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.092 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.099 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:23.113+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:23.102 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:23.200 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:23.214+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:23.231 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.231 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.240 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:23.301 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:23.315+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:23.364 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.364 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.373 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:23.402 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:23.416+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:23.503 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:23.517+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:23.537 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.537 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.546 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:23.604 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:23.618+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:23.684 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.684 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.695 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:23.704 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:23.718+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:23.793 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:23.805 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:23.820+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:23.869 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:23.870 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:23.881 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:23.906 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:23.921+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:23.944 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:23.966 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:23.966 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:23.966 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:23.966 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:23.967 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:23.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:23.969 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:24.007 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:24.021+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:24.021 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:24.021 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:24.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:24.022 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:24.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.022 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:24.022 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 78.713691460999996, 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:24.023 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:24.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:24.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:24.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:24.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:24.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:24.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 73.713188572999996, 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.034 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:24.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:24.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:24.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 67.784517340999997, 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.044 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:24.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:24.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:24.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:24.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.054 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 67.067230870000003, 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:24.055 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:24.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:24.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:24.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.823068656999993, 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.066 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:24.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:24.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:24.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.076 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822537643999993, 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:24.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:29:24.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:29:24.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:29:24.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:29:24.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.087 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822416816, 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:24.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:24.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:24.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:24.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822334502000004, 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.098 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:24.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:24.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:24.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.107 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822241872999996, 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:24.108 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:24.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:24.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:24.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.118 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822159584999994, 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:24.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.128 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822077241000002, 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:24.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:24.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:24.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:24.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.138 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.822002120999997, 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:24.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:24.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:24.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:24.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821923186999996, 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.159 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.159 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:24.159 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821845765000006, 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:24.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:24.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:24.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.169 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821765603000003, 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:24.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:24.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:24.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:24.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.179 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821683789999994, 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821590404000005, 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.190 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:24.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:24.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:24.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821470008000006, 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 64.821369941, 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 23.421096957, 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:24.230 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:24.230 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:29:24.232 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:24.246+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:24.232 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:24.333 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:24.347+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:24.362 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:24.363 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:24.363 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:24.363 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:24.372 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:24.433 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:24.448+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:24.495 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:24.496 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:24.496 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:29:24.496 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:29:24.496 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:24.504 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:24.549 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:24.563+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:24.631 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:24.631 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:29:24.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:29:24.631 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:29:24.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 367.13570464100002, 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.632 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:24.633 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:29:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:29:24.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:24.637 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:29:24.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:24.639 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:29:24.640 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:24.641 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:29:24.642 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 367.13558799800001, 2020-11-16T22:29:24.643 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:24.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.645 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:29:24.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.647 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:29:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:29:24.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:24.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:29:24.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:24.654 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 367.13543962900002, 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:24.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:29:24.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:24.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:29:24.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:29:24.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 367.13465619999999, 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:24.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:24.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:29:24.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:29:24.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:29:24.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:29:24.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.877 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.578619771999996, 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:24.879 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:29:24.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:24.879 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:24.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:24.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:25.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:25.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:25.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:29:25.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:29:25.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:25.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:29:25.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:29:25.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:29:25.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:29:25.293 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:25.294 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 69.866350281999999, 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:25.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:29:25.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:29:25.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:29:25.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:25.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:29:25.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:29:25.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 68.389923586999998, 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:25.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:25.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:29:25.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:25.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:29:25.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:29:25.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:29:25.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.715 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 68.319701121999998, 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:29:25.716 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:25.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:25.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:25.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:29:25.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:29:25.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:29:25.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.925 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:29:25.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:25.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:25.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:26.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:29:26.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:29:26.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.669765390999999, 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:29:26.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:26.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:29:26.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:29:26.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:29:26.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:26.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:29:26.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:29:26.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.342 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:26.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.664018083000002, 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:26.343 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:29:26.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:26.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:29:26.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:29:26.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:26.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:29:26.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:29:26.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:29:26.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.662882078999999, 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:26.557 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:26.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:29:26.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:29:26.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:29:26.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:29:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:29:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:29:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:29:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:29:26.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.972 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.661734983000002, 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:29:26.973 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.974 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:26.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:29:26.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:29:26.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:26.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:27.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:29:27.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:29:27.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:29:27.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:29:27.185 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:27.186 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:27.187 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.655793611000007, 2020-11-16T22:29:27.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:29:27.390 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:27.390 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:27.390 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:27.390 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:27.390 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:27.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:27.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:29:27.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:29:27.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:29:27.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:29:27.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:27.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:29:27.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:29:27.602 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.602 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:27.602 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.602 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.602 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.653448871999998, 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:27.603 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:27.604 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.605 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:27.606 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:29:27.607 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:29:27.607 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:27.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:29:27.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:27.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:29:27.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:27.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:29:27.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:29:27.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:29:27.816 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:27.817 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.648702735000001, 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:29:27.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:27.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:27.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:27.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:29:27.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:27.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:28.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:29:28.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:28.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:28.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:29:28.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:29:28.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:29:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:29:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:29:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.129836191999999, 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:29:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:29:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:29:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:28.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:29:28.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:29:28.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:28.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.443 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.610819426000006, 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:28.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:28.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:29:28.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:29:28.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:28.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:29:28.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:29:28.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:29:28.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:29:28.655 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 65.434012257000006, 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:28.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:28.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:28.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:29:28.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:29:28.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:28.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:29:28.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:29:28.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:28.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:29:28.864 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:28.865 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 65.417320055000005, 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:28.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:28.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:28.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:29.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:29.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:29:29.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:29.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:29:29.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:29:29.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:29:29.076 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 64.388134479000001, 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:29.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:29.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:29:29.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:29:29.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:29:29.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:29:29.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:29.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:29.287 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:29.287 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:29.288 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:29.289 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:29:29.290 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:29.290 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:29.302+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:29.389 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:29.403+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:29.425 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:29.425 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:29.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.425 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:13131 3.0 3:07a02702:::200.0000000c:head [write 2118408~1925 [fadvise_dontneed] in=1925b] snapc 0=[] ondisk+write+known_if_redirected+full_force e101)", 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:29.439390+0000", 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00070577699999999999, 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00072429600000000003, 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:29.426 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439390+0000", 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:29.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439390+0000", 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8867000000000003e-05 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439459+0000", 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6719999999999993e-06 2020-11-16T22:29:29.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439468+0000", 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5492e-05 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439483+0000", 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8549999999999996e-06 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439490+0000", 2020-11-16T22:29:29.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7790999999999999e-05 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439518+0000", 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6848999999999997e-05 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:29.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:29.439575+0000", 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018452599999999999 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:29.432 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:29.433 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:29.433 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:29.433 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:29.435 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:29.476 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:29.490 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:29.504+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:29.497 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:29.497 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:29.497 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:29.498 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:29.498 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:29.500 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:29.500 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:29.574 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:29.574 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:29.584 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:29.591 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:29.604+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:29.691 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:29.705+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:29.767 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:29.767 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:29.777 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:29.821 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:29.834+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:29.893 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:29.906+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:29.914 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:29.914 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:29.924 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:30.008 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:30.021+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:30.090 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.090 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~133s 2020-11-16T22:29:30.095 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.110+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.099 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:30.197 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.211+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.264 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.264 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.273 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:30.317 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.332+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.398 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:30.412+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:30.406 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.406 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.417 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:30.500 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:30.513+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:30.600 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.614+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.604 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.605 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.615 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:30.701 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.715+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.795 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.795 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.802 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:30.816+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:30.806 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:30.902 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:30.917+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:30.933 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:30.933 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:30.942 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:31.003 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.018+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.066 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:31.067 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:31.075 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:31.117 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.131+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.205 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.219+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.240 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:31.241 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:31.249 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:31.290 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:29:31.306 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:31.320+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:31.399 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:31.399 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:31.406 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:31.421+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:31.410 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:31.501 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:31.508 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.522+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.603 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:31.603 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:31.608 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.622+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.614 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:31.646 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:31.667 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:31.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:31.670 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:31.709 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:31.723+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:31.750 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:31.750 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:31.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.442172240000005, 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:31.751 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.752 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:31.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:31.760 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.441669352000005, 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:31.761 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:31.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:31.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 75.512998120000006, 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:31.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:31.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:31.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 74.795711648999998, 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:31.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:31.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:31.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.789 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.551549436000002, 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:31.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:31.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:31.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:31.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.799 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1103 3.6 3:7bf7e7b5:::200.00000006:head [write 1363330~2216 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.551018423000002, 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001, 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211730+0000", 2020-11-16T22:29:31.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4747999999999997e-05 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211795+0000", 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0610000000000002e-06 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211804+0000", 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2578999999999999e-05 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211817+0000", 2020-11-16T22:29:31.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.921e-06 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211822+0000", 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018285692999999999 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230107+0000", 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4356999999999994e-05 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230172+0000", 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101263 2020-11-16T22:29:31.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230273+0000", 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039255050000000001 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234198+0000", 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039752989999999998 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238174+0000", 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1299999999999999e-05 2020-11-16T22:29:31.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238235+0000", 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7583999999999994e-05 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:31.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238323+0000", 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026592310000000001 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:31.809 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550897594999995, 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:31.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:31.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:32.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:32.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:32.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550815280999998, 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.020 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:32.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550722652000005, 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.030 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:32.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:32.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:32.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.240 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550640364000003, 2020-11-16T22:29:32.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:32.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:32.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550558019999997, 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.454 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:32.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:32.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:32.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.666 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550482900000006, 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:32.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:32.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.676 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550403966000005, 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:32.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:32.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:32.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:32.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.687 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.890 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:32.890 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550326544000001, 2020-11-16T22:29:32.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:32.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:32.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:32.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:32.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:32.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:32.900 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550246381999997, 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:32.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:32.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:32.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:33.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550164569000003, 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:33.112 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:33.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:33.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:33.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:33.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:33.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:33.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.550071183, 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:33.324 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:33.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:33.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:33.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:33.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.549950787, 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:33.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:33.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:33.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:33.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 72.549850719999995, 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:33.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:33.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:33.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:33.557 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.149577736000001, 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:33.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:33.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:33.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:33.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:33.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:33.567 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:33.567 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:29:33.569 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:33.582+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:33.570 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:33.669 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:33.683+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:33.672 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:33.707 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:33.708 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:33.708 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:33.717 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:33.800 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:33.814+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:33.817 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:33.837 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:33.838 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:33.838 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:33.838 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:33.838 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:33.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:33.841 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:33.870 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:33.884+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:33.883 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:33.883 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:33.892 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:33.971 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:33.985+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:34.057 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.057 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.066 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:34.072 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.086+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.173 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.187+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.217 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.217 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.228 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:34.274 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.288+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.371 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.371 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.375 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.389+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.377 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:34.476 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.490+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.511 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.512 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.522 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:34.576 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.590+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.649 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.649 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.659 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:34.700 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:34.714+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:34.778 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:34.792+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:34.823 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.824 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.833 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:34.879 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:34.893+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:34.966 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:35.011 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.025+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.081 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.095+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.102 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.103 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.115 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:35.181 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:35.195+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:35.281 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.295+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.306 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.306 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.317 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:35.382 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:35.396+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:35.482 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.497+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.496 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.497 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.508 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:35.570 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:29:35.583 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:35.598+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:35.635 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.635 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.644 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:35.684 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.699+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.785 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:35.800+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:35.808 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.808 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.817 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:35.842 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:35.886 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:35.900+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:35.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~139s 2020-11-16T22:29:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:29:35.943 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:35.943 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:35.953 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:35.990 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:35.994 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:36.008+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:36.010 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:36.012 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:36.013 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:36.087 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:36.101+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14124 4.3 4:cd35a60f:::10000001887.00000000:head [trimtrunc 2290@268934] snapc 1=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.103279+0000", 2020-11-16T22:29:36.346 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0012983109999999999, 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001318084, 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103279+0000", 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103279+0000", 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6166e-05 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103305+0000", 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.31e-06 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103310+0000", 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1235e-05 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103321+0000", 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5010000000000004e-06 2020-11-16T22:29:36.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103327+0000", 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3840000000000001e-05 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103361+0000", 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5466999999999999e-05 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103456+0000", 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017751900000000001 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.352 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:23358 4.3 4.f065acb3 (undecoded) ondisk+write+known_if_redirected e103)", 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.103391+0000", 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001186094, 2020-11-16T22:29:36.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012445710000000001, 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 23358 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.354 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103391+0000", 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103391+0000", 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9120999999999999e-05 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103410+0000", 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6820000000000001e-06 2020-11-16T22:29:36.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103414+0000", 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.571e-06 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103421+0000", 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8779999999999998e-06 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.103424+0000", 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3251999999999999e-05 2020-11-16T22:29:36.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:29:36.359 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:36.360 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:36.360 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:36.359+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:36.446 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:36.460+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:36.530 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:36.530 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:29:36.530 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:29:36.530 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:29:36.531 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:36.541 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:36.584 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:36.599+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:36.648 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:36.662+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:36.680 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:36.680 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:36.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:36.680 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:36.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 91.372283762999999, 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:36.681 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:36.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:36.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:36.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.689 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 86.371780874999999, 2020-11-16T22:29:36.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.691 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:36.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:36.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:36.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.699 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.443109643, 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:36.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:36.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:36.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:36.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.709 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 79.725823172000005, 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:36.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:36.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:36.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.481660958999996, 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:36.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:36.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:36.729 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.481009118000003, 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:36.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:36.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:36.739 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480926804000006, 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:36.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:36.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:36.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:36.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:36.749 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480834174999998, 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:36.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:36.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:36.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:36.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:36.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:36.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.963 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480751886999997, 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:36.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:36.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:36.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.973 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480669543000005, 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:36.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:36.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:36.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:36.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:36.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:36.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:36.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:37.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:37.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:37.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480594422999999, 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:37.185 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:37.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:37.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:37.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:37.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.396 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480515488999998, 2020-11-16T22:29:37.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:37.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:37.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:37.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480438066999994, 2020-11-16T22:29:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:37.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:37.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:37.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:37.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:37.618 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480357905000005, 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480276091999997, 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480182705999994, 2020-11-16T22:29:38.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:38.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:38.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:38.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:38.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:38.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:38.052 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.052 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.254 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.480062309999994, 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:38.255 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:38.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:38.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:38.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:38.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:38.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:38.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:38.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 77.479962243000003, 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:38.265 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:38.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:38.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:38.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:38.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:38.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:38.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:38.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:38.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:38.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:38.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:38.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:38.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 36.079689258999998, 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:38.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:38.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:38.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:38.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:38.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:38.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:38.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:38.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:38.490 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.233716326, 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:38.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:38.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:38.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:38.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:29:38.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:29:38.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:38.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:38.700 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:38.700 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:38.700 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:29:38.701 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:38.703 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:38.715+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:38.703 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:38.801 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:38.816+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:38.837 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:38.838 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:38.838 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:38.838 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:38.848 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:38.852 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:38.873 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:38.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:38.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:38.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:38.902 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:38.917+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:39.003 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:39.017+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:39.019 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.020 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.030 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:39.104 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.118+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.158 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.159 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.167 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:39.209 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.223+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.304 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:39.319+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:39.308 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.308 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.319 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:39.405 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:39.419+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:39.491 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.491 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.501 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:39.505 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.520+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.607 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.621+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.689 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.689 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.700 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:39.745 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.759+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.808 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:39.822+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:39.835 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:39.836 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:39.845 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:39.909 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:39.923+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:39.937 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:29:40.010 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.024+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.017 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.017 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.027 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:40.111 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.125+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.200 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.200 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.210 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:40.251 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.265+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.312 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:40.327+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:40.343 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.344 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.354 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:40.413 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.427+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.514 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.528+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.544 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.545 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.556 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:40.615 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:40.630+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:40.695 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.695 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.705 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:40.746 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:40.761+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:40.818 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:40.832+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:40.834 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.834 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.845 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:40.877 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:40.918 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:40.933+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:40.974 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:40.974 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:40.983 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:41.019 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:41.033+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:41.026 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:41.046 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:41.047 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:41.047 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:41.047 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:41.047 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:41.049 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:41.049 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:41.120 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:41.134+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.154 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 383.65862425300003, 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:29:41.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:29:41.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.165 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 383.65850761000002, 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:29:41.166 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:29:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:29:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:29:41.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:29:41.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:29:41.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:29:41.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:29:41.379 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 383.65835924100003, 2020-11-16T22:29:41.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:41.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:29:41.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:29:41.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:29:41.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:29:41.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.391 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 383.657575812, 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:41.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:29:41.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:29:41.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:29:41.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.401 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 87.101539384000006, 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.402 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:41.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:29:41.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:29:41.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:29:41.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.411 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 86.389269893999995, 2020-11-16T22:29:41.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:29:41.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.419 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.912843198999994, 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:29:41.420 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:29:41.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.427 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.842620733999993, 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:29:41.428 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:29:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.192685002999994, 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:29:41.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:29:41.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.186937694999997, 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.447 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:41.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:29:41.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:29:41.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:29:41.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:41.457 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.185801690999995, 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.458 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:29:41.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:29:41.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:29:41.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.467 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.184654594999998, 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.468 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:41.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:29:41.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:29:41.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:29:41.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.478 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.178713223000003, 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.479 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:41.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:29:41.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:29:41.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:29:41.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.489 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.176368483999994, 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.490 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:41.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:29:41.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:29:41.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:29:41.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:29:41.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:41.500 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 84.171622346999996, 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:29:41.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:29:41.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:29:41.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:29:41.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:29:41.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 83.652755803999995, 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:29:41.511 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:29:41.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:29:41.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:29:41.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 83.133739038000002, 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:29:41.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:29:41.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:29:41.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 81.956931869000002, 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:29:41.527 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:29:41.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:29:41.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:29:41.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.534 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 81.940239667, 2020-11-16T22:29:41.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:29:41.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.542 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 80.911054090999997, 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:29:41.543 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:41.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:29:41.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.550 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.551 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:41.551 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:41.551 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:41.552 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:41.553 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:41.566+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:41.653 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:41.667+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:41.689 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:41.689 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24555 4.3 4:cd35a60f:::10000001887.00000000:head [zero 800506~18593] snapc 1=[] ondisk+write+known_if_redirected e104)", 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:41.702459+0000", 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0021029130000000001, 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021212969999999999, 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24555 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702459+0000", 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702459+0000", 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2929999999999999e-05 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702482+0000", 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7099999999999998e-06 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702487+0000", 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0022e-05 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702497+0000", 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4160000000000003e-06 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702502+0000", 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4972000000000003e-05 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702537+0000", 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6374999999999998e-05 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:41.702634+0000", 2020-11-16T22:29:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000174425 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:41.697 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:41.699 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:41.753 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:41.767+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:41.849 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:29:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:29:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:29:41.850 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:41.854 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:41.868+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:41.861 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:41.949 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:29:41.954 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:41.968+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:42.048 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:42.048 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:42.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:42.048 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:42.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 96.740581274999997, 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:42.049 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:42.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:42.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:42.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:42.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:42.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:42.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:42.057 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 91.740078386999997, 2020-11-16T22:29:42.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.059 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:42.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:42.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:42.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:46.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.221 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.811407154999998, 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:46.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:46.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:46.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:46.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.232 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.094120684000004, 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:46.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:46.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:46.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:46.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:46.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:46.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:46.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.445 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.849958470999994, 2020-11-16T22:29:46.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:46.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:46.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:46.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:46.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.849306630000001, 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.457 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:46.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:46.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:46.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:46.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:46.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:46.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:46.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:46.671 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.849224316000004, 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:46.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:46.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:46.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:46.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:46.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:46.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:46.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:46.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.849131686999996, 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.884 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:46.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:46.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:46.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:46.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.849049398999995, 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:46.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:46.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:46.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:46.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:46.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:46.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:46.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:47.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:47.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:47.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848967055000003, 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.109 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:29:47.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:29:47.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:29:47.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.321 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848891934999997, 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:29:47.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:29:47.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:29:47.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.331 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848813000999996, 2020-11-16T22:29:47.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:47.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:29:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:29:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848735579000007, 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:29:47.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:29:47.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:29:47.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:29:47.352 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:47.353 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848655417000003, 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:29:47.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:29:47.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:29:47.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:29:47.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.363 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848573603999995, 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:29:47.364 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:29:47.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:29:47.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:29:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:29:47.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.374 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848480218000006, 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:29:47.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:29:47.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:29:47.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:29:47.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.384 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848359822000006, 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:29:47.385 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:29:47.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:29:47.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:29:47.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.395 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.848259755000001, 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:29:47.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:29:47.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:29:47.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.405 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.447986770999997, 2020-11-16T22:29:47.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:47.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:29:47.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:29:47.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:47.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:29:47.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 5.6020138380000004, 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:47.417 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:29:47.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:29:47.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:29:47.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:47.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:47.426 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:47.426 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:47.427 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:47.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~144s 2020-11-16T22:29:47.429 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:29:47.429 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:47.441+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:47.430 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:47.528 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:47.542+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:47.564 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:47.564 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:47.564 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:47.564 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:47.573 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:47.613 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:47.629 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:47.643+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:47.634 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:47.635 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:47.635 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:47.635 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:47.635 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:47.637 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:47.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:47.730 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:47.744+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:47.743 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:47.743 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:47.752 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:47.830 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:47.844+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:47.884 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:47.884 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:47.894 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:47.937 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:47.952+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.032 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.046+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.036 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:48.036 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:48.047 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:48.133 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.147+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.233 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:48.233 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:48.234 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.248+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.244 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:48.334 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.348+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.431 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:48.432 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:48.435 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.449+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.443 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:48.537 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.551+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.573 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:48.573 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:48.582 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:48.637 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:48.652+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:48.714 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:48.715 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:48.723 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:48.739 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:48.753+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:48.839 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:48.853+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:48.870 DEBUG:teuthology.orchestra.console:expect before: b'\r\n[Enter `^Ec?\' for help]\r\n\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H\x1b[0m\x1b[2;37;40mInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent GE v1.5.13 \x1b[03;00HPXE 2.1 Build 092 (WfM 2.0) \x1b[04;00HPress Ctrl+S to enter the Setup Menu. \x1b[05;00H \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[04;39H\x1b[01;00H \x1b[04;00HPress Ctrl+S to enter the Setup Menu.. \x1b[04;39H\x1b[04;00H \x1b[05;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H \x1b[03;00HIntel(R) Boot Agent XE v2.1.60 \x1b[04;00HCopyright (C) 1997-2009, Intel Corporation \x1b[05;00H \x1b[06;00HCLIENT MAC ADDR: 0C C4 7A 8F D5 15 GUID: 00000000 0000 0000 0000 0CC47AD8D9E4 \x1b[07;00HDHCP.\\ \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[07;06H\x1b[01;00H \x1b[07;00HDHCP.- \x1b[07;06H\x1b[07;00HDHCP.\\ \x1b[07;06H\x1b[07;00HDHCP.| \x1b[07;06H\x1b[07;00HDHCP./ \x1b[07;06H\x1b[07;00HDHCP.- \x1b[07;06H\x1b[07;00HDHCP.\\ \x1b[07;06H\x1b[07;00HCLIENT IP: 172.21.15.122 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 \x1b[08;00HGATEWAY IP: 172.21.15.254 \x1b[09;00HPXE->EB: !PXE at 96D3:0070, entry point at 96D3:0106 \x1b[10;00H UNDI code segment 96D3:3E70, data segment 8F94:73F0 (574-619kB) \x1b[11;00H UNDI device is PCI 03:00.1, type DIX+802.3 \x1b[12;00H 574kB free base memory after PXE unload \x1b[13;00HiPXE initialising devices... \x1b[13;29H\x1b[13;00HiPXE initialising devices...ok \x1b[17;00H\x1b[0m\x1b[1;37;40miPXE 1.0.0+ (1b67a)\x1b[0m\x1b[2;37;40m -- Open Source Network Boot Firmware -- \x1b[0m\x1b[2;36;40mhttp://ipxe.org\x1b[0m\x1b[2;37;40m \x1b[18;00HFeatures: DNS HTTP HTTPS iSCSI TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT \x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)... \x1b[19;40H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15).... \x1b[19;41H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)..... \x1b[19;42H\x1b[19;00HConfiguring (net0 0c:c4:7a:8f:d5:15)...... ok \x1b[20;00HReceived DHCP answer on interface net0 \x1b[H\x1b[J\x1b[1;1H\x1b[?25l\x1b[m\x1b[H\x1b[J\x1b[1;1H\x1b[20;7H\x1b[mUse the ^ and v keys to change the selection. \n\r Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[4;80H \x1b[7m\x1b[4;1H CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[m\x1b[4;79H\x1b[m\x1b[m\x1b[5;1H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[m\x1b[5;79H\x1b[m\x1b[m\x1b[6;1H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[m\x1b[6;79H\x1b[m\x1b[m\x1b[7;1H \x1b[m\x1b[7;79H\x1b[m\x1b[m\x1b[8;1H \x1b[m\x1b[8;79H\x1b[m\x1b[m\x1b[9;1H \x1b[m\x1b[9;79H\x1b[m\x1b[m\x1b[10;1H \x1b[m\x1b[10;79H\x1b[m\x1b[m\x1b[11;1H \x1b[m\x1b[11;79H\x1b[m\x1b[m\x1b[12;1H \x1b[m\x1b[12;79H\x1b[m\x1b[m\x1b[13;1H \x1b[m\x1b[13;79H\x1b[m\x1b[m\x1b[14;1H \x1b[m\x1b[14;79H\x1b[m\x1b[m\x1b[15;1H \x1b[m\x1b[15;79H\x1b[m\x1b[m\x1b[16;1H \x1b[m\x1b[16;79H\x1b[m\x1b[m\x1b[17;1H \x1b[m\x1b[17;79H\x1b[m\x1b[m\x1b[18;1H \x1b[m\x1b[18;79H\x1b[m\x1b[18;80H \x1b[4;79H\x1b[22;1H The selected entry will be started automatically in 5s. \x1b[4;79H\x1b[03;00H \x1b[04;00H\x1b[0m\x1b[2;30;47m CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[0m\x1b[2;37;40m \x00\x1b[05;00H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[06;00H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H Use the *\x1b[22;1H The selected entry will be started automatically in 4s. \x1b[4;79H and * keys to change the selection. \x1b[22;00H Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[23;00H The selected entry will be started automatically in 4s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 3s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 3s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 2s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 2s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 1s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 1s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 0s. \x1b[4;79H\x1b[?25h\x1b[H\x1b[J\x1b[1;1H\x1b[H\x1b[J\x1b[1;1H\x1b[04;00H \x1b[05;00H \x1b[06;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[01;01H[ 0.000000] Linux version 4.18.0-193.28.1.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Thu Oct 22 00:20:22 UTC 2020\r\n[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200000] x86/fpu: Sture 0x001: \'x87gisters\'\r\n[ 0.000000] x86/fpu: Supporting XSA registers\'\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: \'AVX registers\'\r\n[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256\r\n[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using \'standard\' format.\r\n[ 0.000000] BIOS-provided physical RAM map:\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bbff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078e74fff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x0000000078f24000-0x00000000798adfff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000798ae000-0x0000000079d4afff] ACPI NVS\r\n[ 0.000000] BIOS-e820: [mem 0x0000000079d4b000-0x000000008fffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable\r\n[ 0.000000] NX (Execute Disable) protection: active\r\n[ 0.000000] SMBIOS 3.0 present.\r\n[ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0 12/17/2015\r\n[ 0.000000] tsc: Fast TSC calibration using PIT\r\n[ 0.000000] tsc: Detected 3499.751 MHz processor\r\n[ 0.000000] last_pfn = 0x880000 max_arch_pfn = 0x400000000\r\n[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT \r\n[ 0.000000] last_pfn = 0x78e75 max_arch_pfn = 0x400000000\r\n[ 0.000000] found SMP MP-table at [mem 0x000fda20-0x000fda2f]\r\n[ 0.000000] kexec: Reserving the low 1M of memory for crashkernel\r\n[ 0.000000] Using GB pages for direct mapping\r\n[ 0.000000] RAMDISK: [mem 0x34e8b000-0x3673dfff]\r\n[ 0.000000] ACPI: Early table checksum verification disabled\r\n[ 0.000000] ACPI: RSDP 0x00000000000F0580 000024 (v02 SUPERM)\r\n[ 0.000000] ACPI: XSDT 0x00000000798AE098 0000B4 (v01 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FACP 0x00000000798E15B0 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: DSDT 0x00000000798AE1E8 0333C4 (v02 SUPERM SMCI--MB 01072009 INTL 20091013)\r\n[ 0.000000] ACPI: FACS 0x0000000079D49F80 000040\r\n[ 0.000000] ACPI: APIC 0x00000000798E16C0 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FPDT 0x00000000798E1788 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FIDT 0x00000000798E17D0 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: SPMI 0x00000000798E1870 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000)\r\n[ 0.000000] ACPI: MCFG 0x00000000798E18B0 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097)\r\n[ 0.000000] ACPI: UEFI 0x00000000798E18F0 000042 (v01 SUPERM SMCI--MB 01072009 00000000)\r\n[ 0.000000] ACPI: HPET 0x00000000798E1938 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: WDDT 0x00000000798E1970 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013)\r\n[ 0.000000] ACPI: SSDT 0x00000000798E19B0 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798F8B30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798FB180 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: PRAD 0x00000000798FB1E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: DMAR 0x00000000798FB2F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: HEST 0x00000000798FB3B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: BERT 0x00000000798FB630 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: ERST 0x00000000798FB660 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: EINJ 0x00000000798FB890 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] No NUMA configuration found\r\n[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000087fffffff]\r\n[ 0.000000] NODE_DATA(0) allocated [mem 0x87ffd6000-0x87fffffff]\r\n[ 0.000000] Using crashkernel=auto, the size chosen is a best effort estimation.\r\n[ 0.000000] Reserving 160MB of memory at 1760MB for crashkernel (System RAM: 32653MB)\r\n[ 0.000000] Zone ranges:\r\n[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]\r\n[ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]\r\n[ 0.000000] Normal [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Device empty\r\n[ 0.000000] Movable zone start for each node\r\n[ 0.000000] Early memory node ranges\r\n[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000008afff]\r\n[ 0.000000] node 0: [mem 0x0000000000100000-0x0000000078e74fff]\r\n[ 0.000000] node 0: [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Zeroed struct page in unavailable ranges: 29185 pages\r\n[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff]\r\n[ 0.000000] ACPI: PM-Timer IO Port: 0x408\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])\r\n[ 0.000000] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23\r\n[ 0.000000] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)\r\n[ 0.000000] Using ACPI (MADT) for SMP configuration information\r\n[ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000\r\n[ 0.000000] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0xb000020 (or later)\r\n[ 0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x0008b000-0x000dffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78e75000-0x78f23fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78f24000-0x798adfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x798ae000-0x79d4afff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x79d4b000-0x8fffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x90000000-0xfed1bfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed44fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed45000-0xfeffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]\r\n[ 0.000000] [mem 0x90000000-0xfed1bfff] available for PCI devices\r\n[ 0.000000] Booting paravirtualized kernel on bare hardware\r\n[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns\r\n[ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1\r\n[ 0.000000] percpu: Embedded 53 pages/cpu s180224 r8192 d28672 u262144\r\n[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 8228667\r\n[ 0.000000] Policy zone: Normal\r\n[ 0.000000] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200\r\n[ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://access.redhat.com/ecosystem.\r\n[ 0.000000] Memory: 1856264K/33437692K available (12292K kernel code, 2152K rwdata, 3852K rodata, 2408K init, 6468K bss, 816892K reserved, 0K cma-reserved)\r\n[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1\r\n[ 0.000000] Kernel/User page tables isolation: enabled\r\n[ 0.000000] ftrace: allocating 36345 entries in 142 pages\r\n[ 0.001000] rcu: Hierarchical RCU implementation.\r\n[ 0.001000] rcu: \tRCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.\r\n[ 0.001000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.\r\n[ 0.001000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8\r\n[ 0.001000] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16\r\n[ 0.001000] random: get_random_bytes called from start_kernel+0x36b/0x55b with crng_init=0\r\n[ 0.001000] Console: colour VGA+ 80x25\r\n[ 0.001000] printk: console [tty0] enabled\r\n[ 0.001000] printk: console [ttyS1] enabled\r\n[ 0.001000] ACPI: Core revision 20190703\r\n[ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns\r\n[ 0.001000] APIC: Switch to symmetric I/O mode setup\r\n[ 0.001003] DMAR: Host address width 46\r\n[ 0.002001] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1\r\n[ 0.003004] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de\r\n[ 0.004001] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff\r\n[ 0.005001] DMAR: ATSR flags: 0x0\r\n[ 0.006001] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0\r\n[ 0.007002] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.008001] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.009001] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000\r\n[ 0.010001] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.\r\n[ 0.010002] DMAR-IR: Use \'intremap=no_x2apic_optout\' to override the BIOS setting.\r\n[ 0.012258] DMAR-IR: Enabled IRQ remapping in xapic mode\r\n[ 0.013000] x2apic: IRQ remapping doesn\'t support X2APIC mode\r\n[ 0.014415] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1\r\n[ 0.019001] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x327265fbd4b, max_idle_ns: 440795278449 ns\r\n[ 0.020001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.50 BogoMIPS (lpj=3499751)\r\n[ 0.021001] pid_max: default: 32768 minimum: 301\r\n[ 0.022017] Security Framework initialized\r\n[ 0.023000] Yama: becoming mindful.\r\n[ 0.024002] SELinux: Initializing.\r\n[ 0.029823] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)\r\n[ 0.032443] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)\r\n[ 0.034011] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.035071] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.036190] CPU0: Thermal monitoring enabled (TM1)\r\n[ 0.037035] process: using mwait in idle threads\r\n[ 0.038002] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8\r\n[ 0.039000] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4\r\n[ 0.040001] FEATURE SPEC_CTRL Not Present\r\n[ 0.041000] FEATURE IBPB_SUPPORT Not Present\r\n[ 0.042000] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization\r\n[ 0.043001] Spectre V2 : Mitigation: Full generic retpoline\r\n[ 0.044000] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\r\n[ 0.045000] Speculative Store Bypass: Vulnerable\r\n[ 0.046003] TAA: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.047000] MDS: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.048215] Freeing SMP alternatives memory: 32K\r\n[ 0.052000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1)\r\n[ 0.052061] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver.\r\n[ 0.053001] ... version: 3\r\n[ 0.054000] ... bit width: 48\r\n[ 0.055000] ... generic registers: 4\r\n[ 0.056000] ... value mask: 0000ffffffffffff\r\n[ 0.057000] ... max period: 00007fffffffffff\r\n[ 0.058000] ... fixed-purpose events: 3\r\n[ 0.059000] ... event mask: 000000070000000f\r\n[ 0.060025] rcu: Hierarchical SRCU implementation.\r\n[ 0.061757] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.\r\n[ 0.062014] smp: Bringing up secondary CPUs ...\r\n[ 0.063056] x86: Booting SMP configuration:\r\n[ 0.064002] .... node #0, CPUs: #1 #2 #3 #4\r\n[ 0.066647] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.\r\n[ 0.068001] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details.\r\n[ 0.069072] #5 #6 #7\r\n[ 0.071070] smp: Brought up 1 node, 8 CPUs\r\n[ 0.073001] smpboot: Max logical packages: 1\r\n[ 0.074001] smpboot: Total of 8 processors activated (55996.01 BogoMIPS)\r\n[ 0.278007] node 0 initialised, 7691134 pages in 202ms\r\n[ 0.284304] devtmpfs: initialized\r\n[ 0.285034] x86/mm: Memory block size: 128MB\r\n[ 0.288074] PM: Registering ACPI NVS region [mem 0x798ae000-0x79d4afff] (4837376 bytes)\r\n[ 0.289081] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns\r\n[ 0.300013] futex hash table entries: 2048 (order: 5, 131072 bytes)\r\n[ 0.306060] pinctrl core: initialized pinctrl subsystem\r\n[ 0.312111] NET: Registered protocol family 16\r\n[ 0.317068] audit: initializing netlink subsys (disabled)\r\n[ 0.323010] audit: type=2000 audit(1605565764.323:1): state=initialized audit_enabled=0 res=1\r\n[ 0.332002] cpuidle: using governor menu\r\n[ 0.336022] ACPI FADT declares the system doesn\'t support PCIe ASPM, so disable it\r\n[ 0.344001] ACPI: bus type PCI registered\r\n[ 0.349001] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5\r\n[ 0.355040] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)\r\n[ 0.365002] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820\r\n[ 0.372010] PCI: Using configuration type 1 for base access\r\n[ 0.379015] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages\r\n[ 0.386002] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages\r\n[ 0.394021] cryptd: max_cpu_qlen set to 1000\r\n[ 0.398067] ACPI: Added _OSI(Module Device)\r\n[ 0.403001] ACPI: Added _OSI(Processor Device)\r\n[ 0.408000] ACPI: Added _OSI(3.0 _SCP Extensions)\r\n[ 0.413000] ACPI: Added _OSI(Processor Aggregator Device)\r\n[ 0.418001] ACPI: Added _OSI(Linux-Dell-Video)\r\n[ 0.423000] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)\r\n[ 0.429000] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)\r\n[ 0.458845] ACPI: 4 ACPI AML tables successfully acquired and loaded\r\n[ 0.467935] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored\r\n[ 0.477085] ACPI: Dynamic OEM Table Load:\r\n[ 0.494504] ACPI: Interpreter enabled\r\n[ 0.499011] ACPI: (supports S0 S4 S5)\r\n[ 0.503000] ACPI: Using IOAPIC for interrupt routing\r\n[ 0.508019] HEST: Enabling Firmware First mode for corrected errors.\r\n[ 0.515070] HEST: Table parsing has been initialized.\r\n[ 0.520002] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug\r\n[ 0.530328] ACPI: Enabled 5 GPEs in block 00 to 3F\r\n[ 0.556067] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff])\r\n[ 0.563003] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.573376] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.582461] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.591000] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 0.599025] PCI host bridge to bus 0000:ff\r\n[ 0.604001] pci_bus 0000:ff: root bus resource [bus ff]\r\n[ 0.609008] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000\r\n[ 0.616037] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100\r\n[ 0.622028] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100\r\n[ 0.629030] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000\r\n[ 0.635028] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000\r\n[ 0.641026] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000\r\n[ 0.648027] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000\r\n[ 0.654027] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000\r\n[ 0.660028] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000\r\n[ 0.667026] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000\r\n[ 0.673027] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000\r\n[ 0.679026] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000\r\n[ 0.686030] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000\r\n[ 0.692028] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000\r\n[ 0.699026] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100\r\n[ 0.705027] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000\r\n[ 0.711027] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100\r\n[ 0.718026] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000\r\n[ 0.724026] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000\r\n[ 0.730018] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100\r\n[ 0.737031] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000\r\n[ 0.743055] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000\r\n[ 0.749037] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000\r\n[ 0.756037] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000\r\n[ 0.762037] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000\r\n[ 0.769036] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000\r\n[ 0.775037] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000\r\n[ 0.781029] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000\r\n[ 0.788030] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000\r\n[ 0.794037] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000\r\n[ 0.800037] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000\r\n[ 0.807036] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000\r\n[ 0.813037] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000\r\n[ 0.820029] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000\r\n[ 0.826028] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000\r\n[ 0.832028] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000\r\n[ 0.839029] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000\r\n[ 0.845037] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000\r\n[ 0.851037] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000\r\n[ 0.858037] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000\r\n[ 0.864040] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000\r\n[ 0.870033] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000\r\n[ 0.877028] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000\r\n[ 0.883028] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000\r\n[ 0.890031] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000\r\n[ 0.896028] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000\r\n[ 0.902028] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000\r\n[ 0.909031] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000\r\n[ 0.915032] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000\r\n[ 0.921028] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000\r\n[ 0.928027] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000\r\n[ 0.934027] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000\r\n[ 0.940019] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000\r\n[ 0.947030] pci 0000:ff:1f.0: [8086:6f88] type 00 class 0x088000\r\n[ 0.953027] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000\r\n[ 0.961490] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])\r\n[ 0.968002] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.978185] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.986241] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.995002] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 1.003203] PCI host bridge to bus 0000:00\r\n[ 1.008001] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]\r\n[ 1.015001] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window]\r\n[ 1.022001] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]\r\n[ 1.030001] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window]\r\n[ 1.039001] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window]\r\n[ 1.047000] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window]\r\n[ 1.055001] pci_bus 0000:00: root bus resource [bus 00-fe]\r\n[ 1.060007] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000\r\n[ 1.067103] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400\r\n[ 1.073045] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold\r\n[ 1.080072] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400\r\n[ 1.086044] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold\r\n[ 1.093069] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400\r\n[ 1.099044] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold\r\n[ 1.105070] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000\r\n[ 1.112009] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit]\r\n[ 1.119087] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000\r\n[ 1.125008] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit]\r\n[ 1.133086] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000\r\n[ 1.139008] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit]\r\n[ 1.146085] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000\r\n[ 1.153008] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit]\r\n[ 1.160086] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000\r\n[ 1.166008] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit]\r\n[ 1.173056] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000\r\n[ 1.180008] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit]\r\n[ 1.187085] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000\r\n[ 1.193008] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit]\r\n[ 1.200085] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000\r\n[ 1.207008] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit]\r\n[ 1.214085] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000\r\n[ 1.220081] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000\r\n[ 1.227092] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000\r\n[ 1.233080] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020\r\n[ 1.240006] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff]\r\n[ 1.246093] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000\r\n[ 1.253162] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601\r\n[ 1.259016] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117]\r\n[ 1.265006] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103]\r\n[ 1.271006] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7]\r\n[ 1.277006] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3]\r\n[ 1.283006] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f]\r\n[ 1.289006] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff]\r\n[ 1.295028] pci 0000:00:11.4: PME# supported from D3hot\r\n[ 1.301072] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330\r\n[ 1.307017] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit]\r\n[ 1.314050] pci 0000:00:14.0: PME# supported from D3hot D3cold\r\n[ 1.321064] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000\r\n[ 1.327017] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit]\r\n[ 1.334051] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold\r\n[ 1.341064] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000\r\n[ 1.347017] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit]\r\n[ 1.362051] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold\r\n[ 1.368059] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320\r\n[ 1.375018] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff]\r\n[ 1.381071] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold\r\n[ 1.388070] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400\r\n[ 1.394067] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold\r\n[ 1.401073] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400\r\n[ 1.407068] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold\r\n[ 1.414073] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400\r\n[ 1.420068] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold\r\n[ 1.426077] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320\r\n[ 1.433018] pci 0000:00:1d.0: reg 0x10: [mem 0xfb433000-0xfb4333ff]\r\n[ 1.439071] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold\r\n[ 1.446071] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100\r\n[ 1.452152] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601\r\n[ 1.459013] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077]\r\n[ 1.465005] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063]\r\n[ 1.471005] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057]\r\n[ 1.477005] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043]\r\n[ 1.482005] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f]\r\n[ 1.488005] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff]\r\n[ 1.495029] pci 0000:00:1f.2: PME# supported from D3hot\r\n[ 1.501063] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500\r\n[ 1.507015] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit]\r\n[ 1.514016] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f]\r\n[ 1.520170] acpiphp: Slot [1] registered\r\n[ 1.524013] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 1.530106] pci 0000:02:00.0: [8086:0953] type 00 class 0x010802\r\n[ 1.536012] pci 0000:02:00.0: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit]\r\n[ 1.543016] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref]\r\n[ 1.550003] pci 0000:02:00.0: enabling Extended Tags\r\n[ 1.556074] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 1.561002] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 1.568107] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000\r\n[ 1.575013] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref]\r\n[ 1.582004] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f]\r\n[ 1.588010] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref]\r\n[ 1.596004] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref]\r\n[ 1.603036] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.610016] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.617002] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.628008] pci 0000:03:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.636001] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.647089] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000\r\n[ 1.654013] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref]\r\n[ 1.661004] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f]\r\n[ 1.667010] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref]\r\n[ 1.675004] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref]\r\n[ 1.682036] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.689015] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.697001] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.707008] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.715001] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.726080] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 1.732001] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 1.738001] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 1.745002] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 1.753028] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 1.759053] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400\r\n[ 1.765037] pci 0000:05:00.0: enabling Extended Tags\r\n[ 1.771040] pci 0000:05:00.0: supports D1 D2\r\n[ 1.775002] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.782086] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 1.788002] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 1.794002] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.801048] pci_bus 0000:06: extended config space not accessible\r\n[ 1.808030] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000\r\n[ 1.814023] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff]\r\n[ 1.821012] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff]\r\n[ 1.827012] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f]\r\n[ 1.833098] pci 0000:06:00.0: supports D1 D2\r\n[ 1.838001] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.845058] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 1.850006] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 1.857004] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.864069] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000\r\n[ 1.870029] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff]\r\n[ 1.877018] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f]\r\n[ 1.883009] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff]\r\n[ 1.889113] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.896035] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.904001] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.915021] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.923001] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 1.933100] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000\r\n[ 1.940027] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff]\r\n[ 1.946018] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f]\r\n[ 1.952009] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff]\r\n[ 1.959062] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.966034] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.973001] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.984021] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.992001] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 2.003110] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 2.008002] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 2.015002] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 2.022004] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08\r\n[ 2.029413] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)\r\n[ 2.037033] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15)\r\n[ 2.044031] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)\r\n[ 2.051031] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)\r\n[ 2.059031] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 2.068031] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 2.076031] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 2.085030] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 2.094092] iommu: Default domain type: Passthrough \r\n[ 2.099018] pci 0000:06:00.0: vgaarb: setting as boot VGA device\r\n[ 2.100000] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none\r\n[ 2.115001] pci 0000:06:00.0: vgaarb: bridge control possible\r\n[ 2.121000] vgaarb: loaded\r\n[ 2.124059] SCSI subsystem initialized\r\n[ 2.128014] ACPI: bus type USB registered\r\n[ 2.133010] usbcore: registered new interface driver usbfs\r\n[ 2.138004] usbcore: registered new interface driver hub\r\n[ 2.144017] usbcore: registered new device driver usb\r\n[ 2.149017] pps_core: LinuxPPS API ver. 1 registered\r\n[ 2.155000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti \r\n[ 2.164001] PTP clock support registered\r\n[ 2.169014] EDAC MC: Ver: 3.0.0\r\n[ 2.172100] PCI: Using ACPI for IRQ routing\r\n[ 2.182000] NetLabel: Initializing\r\n[ 2.186000] NetLabel: domain hash size = 128\r\n[ 2.190000] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO\r\n[ 2.196009] NetLabel: unlabeled traffic allowed by default\r\n[ 2.202034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0\r\n[ 2.209001] hpet0: 8 comparators, 64-bit 14.318180 MHz counter\r\n[ 2.217019] clocksource: Switched to clocksource tsc-early\r\n[ 2.231146] VFS: Disk quotas dquot_6.6.0\r\n[ 2.235408] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)\r\n[ 2.242653] pnp: PnP ACPI init\r\n[ 2.246333] system 00:01: [io 0x0500-0x057f] has been reserved\r\n[ 2.252578] system 00:01: [io 0x0400-0x047f] has been reserved\r\n[ 2.258826] system 00:01: [io 0x0580-0x059f] has been reserved\r\n[ 2.265075] system 00:01: [io 0x0600-0x061f] has been reserved\r\n[ 2.271326] system 00:01: [io 0x0880-0x0883] has been reserved\r\n[ 2.277574] system 00:01: [io 0x0800-0x081f] has been reserved\r\n[ 2.283826] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved\r\n[ 2.290769] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved\r\n[ 2.297710] system 00:01: [mem 0xff000000-0xffffffff] has been reserved\r\n[ 2.304654] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved\r\n[ 2.311594] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved\r\n[ 2.318539] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved\r\n[ 2.325482] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved\r\n[ 2.332568] system 00:02: [io 0x0a00-0x0a0f] has been reserved\r\n[ 2.338819] system 00:02: [io 0x0a10-0x0a1f] has been reserved\r\n[ 2.345068] system 00:02: [io 0x0a20-0x0a2f] has been reserved\r\n[ 2.351317] system 00:02: [io 0x0a30-0x0a3f] has been reserved\r\n[ 2.357565] system 00:02: [io 0x0a40-0x0a4f] has been reserved\r\n[ 2.364483] pnp: PnP ACPI: found 5 devices\r\n[ 2.374216] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns\r\n[ 2.383717] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000\r\n[ 2.403451] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.411880] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 2.417176] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 2.422467] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 2.429592] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.437758] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.446270] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.454522] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.463121] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.471285] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.479797] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.488046] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.496648] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.504811] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.513323] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.521573] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.530173] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.538336] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.546850] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.555099] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.563699] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 2.568991] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 2.575416] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 2.582534] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.590873] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 2.596171] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 2.601470] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 2.607898] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.615029] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 2.620581] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 2.627006] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.634128] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref]\r\n[ 2.642471] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref]\r\n[ 2.650896] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref]\r\n[ 2.659234] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref]\r\n[ 2.667660] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 2.672949] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 2.679376] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 2.686491] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.694832] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc\r\n[ 2.704471] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]\r\n[ 2.710975] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window]\r\n[ 2.717484] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]\r\n[ 2.724690] pci_bus 0000:00: resource 7 [mem 0xfedb0000-0xfedb000f window]\r\n[ 2.731892] pci_bus 0000:00: resource 8 [mem 0xfedc0000-0xfedc000f window]\r\n[ 2.739095] pci_bus 0000:00: resource 9 [mem 0x90000000-0xfbffbfff window]\r\n[ 2.746297] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff]\r\n[ 2.752891] pci_bus 0000:03: resource 0 [io 0xe000-0xefff]\r\n[ 2.758794] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff]\r\n[ 2.765390] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.773208] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff]\r\n[ 2.779108] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.785707] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff]\r\n[ 2.791607] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.798205] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff]\r\n[ 2.804105] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff]\r\n[ 2.810703] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.818629] NET: Registered protocol family 2\r\n[ 2.823415] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes)\r\n[ 2.832093] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)\r\n[ 2.840328] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)\r\n[ 2.847467] TCP: Hash tables configured (established 262144 bind 65536)\r\n[ 2.854459] UDP hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.861034] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.868079] NET: Registered protocol family 1\r\n[ 2.872769] NET: Registered protocol family 44\r\n[ 2.894130] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x6c5 took 15993 usecs\r\n[ 2.914123] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x6c5 took 11700 usecs\r\n[ 2.922125] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes\r\n[ 2.928906] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]\r\n[ 2.937874] Unpacking initramfs...\r\n[ 3.198271] Freeing initrd memory: 25292K\r\n[ 3.211018] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\r\n[ 3.217792] software IO TLB: mapped [mem 0x6a000000-0x6e000000] (64MB)\r\n[ 3.233450] Initialise system trusted keyrings\r\n[ 3.238232] Key type blacklist registered\r\n[ 3.242594] workingset: timestamp_bits=36 max_order=23 bucket_order=0\r\n[ 3.250314] zbud: loaded\r\n[ 3.253401] pstore: using deflate compression\r\n[ 3.258264] Platform Keyring initialized\r\n[ 3.320552] NET: Registered protocol family 38\r\n[ 3.325324] Key type asymmetric registered\r\n[ 3.329749] Asymmetric key parser \'x509\' registered\r\n[ 3.334957] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)\r\n[ 3.342977] io scheduler mq-deadline registered\r\n[ 3.347844] io scheduler kyber registered\r\n[ 3.352228] io scheduler bfq registered\r\n[ 3.356476] atomic64_test: passed for x86-64 platform with CX8 and with SSE\r\n[ 3.364026] pcieport 0000:00:01.0: Signaling with IRQ 25\r\n[ 3.369816] pcieport 0000:00:02.0: Signaling with IRQ 27\r\n[ 3.375597] pcieport 0000:00:03.0: Signaling with IRQ 29\r\n[ 3.381358] pcieport 0000:00:1c.0: Signaling with IRQ 30\r\n[ 3.387135] pcieport 0000:00:1c.2: Signaling with IRQ 31\r\n[ 3.392899] pcieport 0000:00:1c.4: Signaling with IRQ 32\r\n[ 3.398623] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4\r\n[ 3.406194] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0\r\n[ 3.415148] ACPI: Power Button [PWRB]\r\n[ 3.419162] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1\r\n[ 3.427162] ACPI: Power Button [PWRF]\r\n[ 3.432239] ERST: Error Record Serialization Table (ERST) support is initialized.\r\n[ 3.440348] pstore: Registered erst as persistent store backend\r\n[ 3.446750] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.\r\n[ 3.454827] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled\r\n[ 3.482005] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A\r\n[ 3.510531] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A\r\n[ 3.518736] Non-volatile memory driver v1.3\r\n[ 3.523624] rdac: device handler registered\r\n[ 3.528182] hp_sw: device handler registered\r\n[ 3.532781] emc: device handler registered\r\n[ 3.537231] alua: device handler registered\r\n[ 3.541761] libphy: Fixed MDIO Bus: probed\r\n[ 3.546238] ehci_hcd: USB 2.0 \'Enhanced\' Host Controller (EHCI) Driver\r\n[ 3.553093] ehci-pci: EHCI PCI platform driver\r\n[ 3.557975] ehci-pci 0000:00:1a.0: EHCI Host Controller\r\n[ 3.563590] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1\r\n[ 3.571588] ehci-pci 0000:00:1a.0: debug port 2\r\n[ 3.580355] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported\r\n[ 3.587476] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000\r\n[ 3.600038] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00\r\n[ 3.606138] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.614998] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.622812] usb usb1: Product: EHCI Host Controller\r\n[ 3.628019] usb usb1: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.635925] usb usb1: SerialNumber: 0000:00:1a.0\r\n[ 3.640964] hub 1-0:1.0: USB hub found\r\n[ 3.645042] hub 1-0:1.0: 2 ports detected\r\n[ 3.649531] ehci-pci 0000:00:1d.0: EHCI Host Controller\r\n[ 3.655117] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2\r\n[ 3.663117] ehci-pci 0000:00:1d.0: debug port 2\r\n[ 3.671885] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported\r\n[ 3.679009] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000\r\n[ 3.692009] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00\r\n[ 3.698106] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.706967] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.714783] usb usb2: Product: EHCI Host Controller\r\n[ 3.727631] usb usb2: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.735533] usb usb2: SerialNumber: 0000:00:1d.0\r\n[ 3.740560] hub 2-0:1.0: USB hub found\r\n[ 3.744643] hub 2-0:1.0: 2 ports detected\r\n[ 3.749065] ohci_hcd: USB 1.1 \'Open\' Host Controller (OHCI) Driver\r\n[ 3.755575] ohci-pci: OHCI PCI platform driver\r\n[ 3.760352] uhci_hcd: USB Universal Host Controller Interface driver\r\n[ 3.767146] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.772751] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3\r\n[ 3.781790] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810\r\n[ 3.791526] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported\r\n[ 3.798771] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.807632] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.815446] usb usb3: Product: xHCI Host Controller\r\n[ 3.820653] usb usb3: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.828562] usb usb3: SerialNumber: 0000:00:14.0\r\n[ 3.833591] hub 3-0:1.0: USB hub found\r\n[ 3.837679] hub 3-0:1.0: 15 ports detected\r\n[ 3.843572] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.849152] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4\r\n[ 3.857150] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed\r\n[ 3.863848] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 4.18\r\n[ 3.872707] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.880521] usb usb4: Product: xHCI Host Controller\r\n[ 3.885729] usb usb4: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.893636] usb usb4: SerialNumber: 0000:00:14.0\r\n[ 3.898659] hub 4-0:1.0: USB hub found\r\n[ 3.902747] hub 4-0:1.0: 6 ports detected\r\n[ 3.907872] usbcore: registered new interface driver usbserial_generic\r\n[ 3.914729] usbserial: USB Serial support registered for generic\r\n[ 3.921088] i8042: PNP: No PS/2 controller found.\r\n[ 3.926164] mousedev: PS/2 mouse device common for all mice\r\n[ 3.932128] rtc_cmos 00:00: RTC can wake from S4\r\n[ 3.937217] rtc_cmos 00:00: registered as rtc0\r\n[ 3.941999] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram, hpet irqs\r\n[ 3.950261] intel_pstate: Intel P-state driver initializing\r\n[ 3.956975] hidraw: raw HID events driver (C) Jiri Kosina\r\n[ 3.962731] usbcore: registered new interface driver usbhid\r\n[ 3.968636] usbhid: USB HID core driver\r\n[ 3.972868] drop_monitor: Initializing network drop monitor service\r\n[ 3.979502] Initializing XFRM netlink socket\r\n[ 3.984172] NET: Registered protocol family 10\r\n[ 3.989130] Segment Routing with IPv6\r\n[ 3.993129] NET: Registered protocol family 17\r\n[ 3.997992] mpls_gso: MPLS GSO support\r\n[ 4.002426] core: Using 22 MCE banks\r\n[ 4.006361] microcode: sig=0x406f1, pf=0x1, revision=0xb000010\r\n[ 4.012526] usb 1-1: new high-speed USB device number 2 using ehci-pci\r\n[ 4.012603] microcode: Microcode Update Driver: v2.2.\r\n[ 4.019560] resctrl: L3 allocation detected\r\n[ 4.029453] resctrl: L3DATA allocation detected\r\n[ 4.034320] resctrl: L3CODE allocation detected\r\n[ 4.039180] resctrl: L3 monitoring detected\r\n[ 4.043698] AVX2 version of gcm_enc/dec engaged.\r\n[ 4.048641] AES CTR mode by8 optimization enabled\r\n[ 4.078398] sched_clock: Marking stable (4078392770, 0)->(5626693585, -1548300815)\r\n[ 4.086767] registered taskstats version 1\r\n[ 4.091220] Loading compiled-in X.509 certificates\r\n[ 4.114006] usb 2-1: new high-speed USB device number 2 using ehci-pci\r\n[ 4.122874] Loaded X.509 cert \'CentOS kernel signing key: dbeb60232b9048c59969054d0225be669c9ec38b\'\r\n[ 4.132615] Loaded X.509 cert \'CentOS Linux Driver update signing key: 29bd4c0d06d2e9911044b5dc973309139b51d6d5\'\r\n[ 4.143437] Loaded X.509 cert \'CentOS Linux kpatch signing key: b49f086205909dc4da2cfa99376fb191d2f09e78\'\r\n[ 4.153647] zswap: loaded using pool lzo/zbud\r\n[ 4.163045] Key type big_key registered\r\n[ 4.169520] Key type encrypted registered\r\n[ 4.173871] usb 3-12: new high-speed USB device number 2 using xhci_hcd\r\n[ 4.180829] ima: No TPM chip found, activating TPM-bypass!\r\n[ 4.186915] usb 1-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05\r\n[ 4.189028] ima: Allocated hash algorithm: sha1\r\n[ 4.195708] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.200600] evm: Initialising EVM extended attributes\r\n[ 4.213553] evm: security.selinux\r\n[ 4.213815] hub 1-1:1.0: USB hub found\r\n[ 4.217213] evm: security.ima\r\n[ 4.217213] evm: security.capability\r\n[ 4.217214] evm: HMAC attrs: 0x1\r\n[ 4.224796] hub 1-1:1.0: 6 ports detected\r\n[ 4.232107] rtc_cmos 00:00: setting system clock to 2020-11-16 22:29:30 UTC (1605565770)\r\n[ 4.246010] Freeing unused decrypted memory: 2040K\r\n[ 4.251486] Freeing unused kernel memory: 2408K\r\n[ 4.255009] tsc: Refined TSC clocksource calibration: 3499.995 MHz\r\n[ 4.258014] Write protecting the kernel read-only data: 18432k\r\n[ 4.262859] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734c8180a, max_idle_ns: 440795326857 ns\r\n[ 4.279770] clocksource: Switched to clocksource tsc\r\n[ 4.285253] Freeing unused kernel memory: 2020K\r\n[ 4.290171] Freeing unused kernel memory: 244K\r\n[ 4.297299] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05\r\n[ 4.306075] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.306318] hub 2-1:1.0: USB hub found\r\n[ 4.317695] hub 2-1:1.0: 8 ports detected\r\n[ 4.323455] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 4.343274] usb 3-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00\r\n[ 4.354738] usb 3-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.362882] hub 3-12:1.0: USB hub found\r\n[ 4.367075] systemd[1]: Detected architecture x86-64.\r\n[ 4.367117] hub 3-12:1.0: 4 ports detected\r\n[ 4.372454] systemd[1]: Running in initial RAM disk.\r\n\r\nWelcome to \x1b[0;34mCentOS Linux 8 (Core) dracut-049-27.git20190906.el8 (Initramfs)\x1b[0m!\r\n\r\n[ 4.393126] systemd[1]: Set hostname to .\r\n[ 4.428629] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.435637] systemd[1]: Listening on udev Kernel Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Kernel Socket.\r\n[ 4.448158] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.455131] systemd[1]: Reached target Swap.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\n[ 4.465086] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.472094] systemd[1]: Listening on Journal Socket (/dev/log).\r\n[\x1b[0;32m OK \x1b[0m] Listening on Journal Socket (/dev/log).\r\n[ 4.485146] systemd[1]: Listening on Journal Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on Journal Socket.\r\n[ 4.498067] systemd[1]: Starting Create list of required static device nodes for the current kernel...\r\n Starting Create list of required st\xe2\x80\xa6ce nodes for the current kernel...\r\n[ 4.517101] systemd[1]: Reached target Timers.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\n Starting Setup Virtual Console...\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\n Starting Journal Service...\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\n Starting Create Volatile Files and Directories...\r\n Starting Apply Kernel Variables...\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directories.\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\n Starting dracut cmdline hook...\r\n Starting Create Static Device Nodes in /dev...\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\n[ 4.677007] usb 3-12.1: new low-speed USB device number 3 using xhci_hcd\r\n[\x1b[0;32m OK \x1b[0m] Started dracut cmdline hook.\r\n Starting dracut pre-udev hook...\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-udev hook.\r\n Starting udev Kernel Device Manager...\r\n[ 4.789183] usb 3-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00\r\n[ 4.798221] usb 3-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.823894] i HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.0/0003:0557:2419.0001/input/input2\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\n[ 4.902092] heric 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0\r\n[ 4.927646] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.1/0003:0557:2419.0002/input/input3\r\n[ 4.952046] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\n Starting udev Coldplug all Devices...\r\n Mounting Kernel Configuration File System...\r\n[\x1b[0;32m OK \x1b[0m] Mounted Kernel Configuration File System.\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\n Starting dracut initqueue hook...\r\n Starting Show Plymouth Boot Screen...\r\n[ 5.160298] dca service started, version 1.12.1\r\n[\x1b[0;32m OK \x1b[0m] Started Show Plymouth Boot Screen.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Pas[ 5.193713] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k-rh8.2.0\r\nsword Requests t[ 5.204218] ixgbe: Copyright (c) 1999-2016 Intel Corporation.\r\no Plymouth Direc[ 5.211320] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k\r\ntory Watch.\r\r\n[\x1b[ 5.219431] igb: Copyright (c) 2007-2014 Intel Corporation.\r\n[0;32m OK \x1b[0m] Reached target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Bas[ 5.231814] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode\r\nic System.\r\r\n[ 5.241677] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 5.273293] nvme nvme0: pci function 0000:02:00.0\r\n[ 5.295891] [drm] Using P2A bridge for configuration\r\n[ 5.301199] [drm] AST 2400 detected\r\n[ 5.305039] [drm] Analog VGA only\r\n[ 5.308720] [drm] dram MCLK=408 Mhz type=1 bus_width=16 size=01000000\r\n[ 5.315618] igb 0000:07:00.0: added PHC on eth0\r\n[ 5.320504] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.327734] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:e4\r\n[ 5.335304] igb 0000:07:00.0: eth0: PBA No: 050B00-000\r\n[ 5.340781] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)\r\n[ 5.350122] scsi host0: ahci\r\n[ 5.354023] scsi host1: ahci\r\n[ 5.364371] scsi host2: ahci\r\n[ 5.372728] [TTM] Zone kernel: Available graphics memory: 16326418 KiB\r\n[ 5.379681] [TTM] Zone dma32: Available graphics memory: 2097152 KiB\r\n[ 5.383664] nvme nvme0: 8/0/0 default/read/poll queues\r\n[ 5.386550] [TTM] Initializing pool allocator\r\n[ 5.405421] scsi host3: ahci\r\n[ 5.408721] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 43\r\n[ 5.424380] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 43\r\n[ 5.432387] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 43\r\n[ 5.440387] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 43\r\n[ 5.464013] [TTM] Initializing DMA pool allocator\r\n[ 5.483845] fbcon: astdrmfb (fb0) is primary device\r\n[ 5.568540] Console: switching to colour frame buffer device 128x48\r\n[ 5.630302] ast 0000:06:00.0: fb0: astdrmfb frame buffer device\r\n[ 5.641711] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode\r\n[ 5.649919] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 5.673023] [drm] Initialized ast 0.1.0 20120228 for 0000:06:00.0 on minor 0\r\n[ 5.700435] igb 0000:07:00.1: added PHC on eth1\r\n[ 5.704995] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.711900] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:e5\r\n[ 5.719151] igb 0000:07:00.1: eth1: PBA No: 050B00-000\r\n[ 5.724316] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), [ 5.736620] scsi host4: ahci\r\n[ 5.742204] scsi host5: ahci\r\n[ 5.749955] scsi host6: ahci\r\n[ 5.756132] ata4: SATA link down (SStatus 0 SControl 300)\r\n[ 5.761572] ata3: SATA link down (SStatus 0 SControl 300)\r\n[ 5.767017] ata2: SATA link down (SStatus 0 SControl 300)\r\n[ 5.770635] scsi host7: ahci\r\n[ 5.772479] ata1: SATA link down (SStatus 0 SControl 300)\r\n[ 5.791212] scsi host8: ahci\r\n[ 5.804248] scsi host9: ahci\r\n[ 5.807230] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 63\r\n[ 5.814667] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 63\r\n[ 5.822093] ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 63\r\n[ 5.830270] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 63\r\n[ 5.838445] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 63\r\n[ 5.8465ta10: SATA max Ub432000 port 0xf[ 5.865629] igb 0000:07:00.0 eno1: renamed from eth0\r\n[ 5.890519] igb 0000:07:00.1 eno2: renamed from eth1\r\n[ 6.164078] ata7: SATA link down (SStatus 0 SControl 300)\r\n[ 6.170232] ata10: SATA link down (SStatus 0 SControl 300)\r\n[ 6.176473] ata8: SATA link down (SStatus 0 SControl 300)\r\n[ 6.182301] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300)\r\n[ 6.188921] ata6: SATA link down (SStatus 0 SControl 300)\r\n[ 6.194753] ata9: SATA link down (SStatus 0 SControl 300)\r\n[ 6.276525] ata5.00: NCQ Send/Recv Log not supported\r\n[ 6.277257] random: crng init done\r\n[ 6.281906] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133\r\n[ 6.285720] random: 7 urandom warning(s) missed due to ratelimiting\r\n[ 6.299036] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 32), AA\r\n[ 6.307812] ata5.00: NCQ Send/Recv Log not supported\r\n[ 6.313170] ata5.00: configured for UDMA/133\r\n[ 6.317916] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5\r\n[ 6.332393] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)\r\n[ 6.340471] sd 4:0:0:0: [sda] Write Protect is off\r\n[ 6.345666] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn\'t support DPO or FUA\r\n[ 6.370179] sda: sda1\r\n[ 6.372247] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.373533] sd 4:0:0:0: [sda] Attached SCSI disk\r\n[ 6.384325] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.397776] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF\r\n[ 6.404399] ixgbe 0000:03:00.0: 0c:c4:7a:8f:d5:14\r\n[ 6.412534] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection\r\n[ 6.419540] libphy: ixgbe-mdio: probed\r\n[\x1b[0;32m OK \x1b[0m] Found device ST1000NM0033-9ZM173 1.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root Device.\r\r\n[ 6.580253] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.590955] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.599284] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF\r\n[ 6.606731] ixgbe 0000:03:00.1: 0c:c4:7a:8f:d5:15\r\n[ 6.614796] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection\r\n[ 6.621752] libphy: ixgbe-mdio: probed\r\n[ 6.626784] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0\r\n[ 6.643129] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1\r\n[\x1b[0;32m OK \x1b[0m] Started dracut initqueue hook.\r\r\n Starting File System Check on /dev/\xe2\x80\xa68e7-5b1a-4776-884c-191bd8ad39f5...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started File System Check on /dev/d\xe2\x80\xa6f28e7-5b1a-4776-884c-191bd8ad39f5.\r\r\n Mounting /sysroot...\r\r\n[ 6.806065] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)\r\n[\x1b[0;32m OK \x1b[0m] Mounted /sysroot.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root File System.\r\r\n Starting Reload Configuration from the Real Root...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Reload Configuration from the Real Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Default Target.\r\r\n Starting dracut pre-pivot and cleanup hook...\r\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-pivot and cleanup hook.\r\r\n Starting Cleaning Up and Shutting Down Daemons...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-pivot and cleanup hook.\r\r\n Starting Setup Virtual Console...\r\r\n Starting Plymouth switch root service...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut initqueue hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Default Target.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Basic System.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Apply Kernel Variables.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Swap.\r\r\n Stopping udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Sockets.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root Device.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Volatile Files and Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Local File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Slices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Kernel Device Manager.\r\r\n Stopping Hardware RNG Entropy Gatherer Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Static Device Nodes in /dev.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create list of required sta\xe2\x80\xa6vice [ 7.220870] systemd-journald[319]: Received SIGTERM from PID 1 (systemd).\r\nnodes for the current kernel.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-udev hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut cmdline hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Kernel Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Control Socket.\r\r\n Starting Cleanup udevd DB...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Hardware RNG Entropy Gatherer Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleaning Up and Shutting Down Daemons.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleanup udevd DB.\r\r\n[ 7.305635] printk: systemd: 21 output lines suppressed due to ratelimiting\r\n[\x1b[0;32m OK \x1b[0m] Started Plymouth switch root service.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Switch Root.\r\r\n Starting Switch Root...\r\r\n[ 8.127245] SELinux: policy capability network_peer_controls=1\r\n[ 8.133504] SELinux: policy capability open_perms=1\r\n[ 8.138792] SELinux: policy capability extended_socket_class=1\r\n[ 8.145028] SELinux: policy capability always_check_network=0\r\n[ 8.151169] SELinux: policy capability cgroup_seclabel=1\r\n[ 8.156875] SELinux: policy capability nnp_nosuid_transition=1\r\n[ 8.178655] audit: type=1403 audit(1605565774.448:2): auid=4294967295 ses=4294967295 lsm=selinux res=1\r\n[ 8.180585] systemd[1]: Successfully loaded SELinux policy in 435.138ms.\r\n[ 8.398684] systemd[1]: Relabelled /dev, /run and /sys/fs/cgroup in 21.452ms.\r\n[ 8.415353] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 8.449155] systemd[1]: Detected architecture x86-64.\r\n\r\r\nWelcome to \x1b[0;31mCentOS Linux 8 (Core)\x1b[0m!\r\r\n\r\r\n[ 8.460809] systemd[1]: Set hostname to .\r\n[ 9.356884] systemd[1]: Stopped Switch Root.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Switch Root.\r\r\n[ 9.361911] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart.\r\n[ 9.361971] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.\r\n[ 9.362048] systemd[1]: Stopped Journal Service.\r\n[ 9.404700] systemd[1]: Starting Journal Service...\r\n[ 9.426597] systemd[1]: Listening on Device-mapper event daemon FIFOs.\r\n[ 9.456306] systemd[1]: Reached target Swap.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Journal Service.\r\r\n Starting Journal Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Device-mapper event daemon FIFOs.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Process Core Dump Socket.\r\r\n Starting Remount Root and Kernel File Systems...\r\r\n Starting Create list of required st\xe2\x80\xa6ce nodes for the current kernel...\r\r\n Mounting Huge Pages File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-sshd\\x2dkeygen.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on RPCbind Server Activation Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Kernel Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on LVM2 poll daemon socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Set up automount Arbitrary Executab\xe2\x80\xa6rmats File System Automount Point.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-getty.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on initctl Compatibility Named Pipe.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-serial\\x2dgetty.slice.\r\r\n Starting udev Coldplug all Devices...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6mirror@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Password Requests to Wall Directory Watch.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local Encrypted Volumes.\r\r\n Mounting Kernel Debug File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mon@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target al[ 9.659550] EXT4-fs (sda1): re-mounted. Opts: (null)\r\nlowing\xe2\x80\xa6ph-mds@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6adosgw@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-osd@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mgr@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice User and Session Slice.\r\r\n Mounting POSIX Message Queue File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target RPC Port Mapper.\r\r\n Starting Apply Kernel Variables...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Switch Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root File System.\r\r\n Starting Setup Virtual Console...\r\r\n Starting Monitoring of LVM2 mirrors\xe2\x80\xa6ng dmeventd or progress polling...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Remount Root and Kernel File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted Huge Pages File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Mou[ 9.830393] systemd-journald[1428]: Received request to flush runtime journal from PID 1\r\nnted Kernel Debug File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted POSIX Message Queue File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\r\n Starting Load/Save Random Seed...\r\r\n Starting Create Static Device Nodes in /dev...\r\r\n Starting Flush Journal to Persistent Storage...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load/Save Random Seed.\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Flush Journal to Persistent Storage.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\r\n Starting udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\r\n[ 10.424159] ioatdma: Intel(R) QuickData Technology Driver 5.00\r\n[ 10.439319] igb 0000:07:00.0: DCA enabled\r\n[ 10.444395] igb 0000:07:00.1: DCA enabled\r\n[ 10.455226] mei_me 0000:00:16.0: Device doesn\'t have valid ME Interface\r\n[ 10.495913] IPMI message handler: version 39.2\r\n[ 10.504542] ipmi device interface\r\n[ 10.517494] ipmi_si: IPMI System Interface driver\r\n[ 10.522658] ipmi_si dmi-ipmi-si.0: probing via SMBIOS\r\n[ 10.528252] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0\r\n[ 10.536037] ipmi_si: Adding SMBIOS-specified kcs state machine\r\n[ 10.542322] ipmi_si IPI0001:00: probing via ACPI\r\n[ 10.547387] ipmi_si IPI0001:00: [io 0x0ca2] regsize 1 spacing 1 irq 0\r\n[ 10.554339] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI\r\n[ 10.559622] power_meter ACPI000D:00: Found ACPI power meter.\r\n[ 10.563660] ipmi_si: Adding ACPI-specified kcs state machine\r\n[ 10.570030] power_meter ACPI000D:00: Ignoring unsafe software power cap!\r\n[ 10.576389] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0\r\n[ 10.643245] input: PC Speaker as /devices/platform/pcspkr/input/input4\r\n[ 10.650572] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-lvm2\\x2dpvscan.slice.\r\r\n[ 10.657063] sd 4:0:0:0: Attached scsi generic sg0 type 0\r\n[ 10.667051] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed.\r\n[ 10.690046] ipmi_si IPI0001:00: Found new BMC (man_id: 0x002a7c, prod_id: 0x0833, dev_id: 0x20)\r\n[ 10.719545] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer\r\n[ 10.719545] RAPL PMU: hw unit of domain package 2^-14 Joules\r\n[ 10.719546] RAPL PMU: hw unit of domain dram 2^-16 Joules\r\n[ 10.722827] device-mapper: uevent: version 1.0.3\r\n[ 10.722893] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com\r\n Starting LVM event activation on device 259:0...\r\r\n[ 10.842697] ipmi_si IPI0001:00: IPMI kcs interface initialized\r\n[ 10.853062] ipmi_ssif: IPMI SSIF Interface driver\r\n[ 11.048206] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:ff:12.0 (INTERRUPT)\r\n[ 11.060076] EDAC sbridge: Ver: 1.1.2 \r\n[ 11.175581] iTCO_vendor_support: vendor-support=0\r\n[ 11.182340] intel_rapl_common: Found RAPL domain package\r\n[ 11.182344] intel_rapl_common: Found RAPL domain dram\r\n[ 11.182346] intel_rapl_common: DRAM domain energy unit 15300pj\r\n[ 11.204384] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11\r\n[ 11.204541] iTCO_wdt: Found a Wellsburg TCO device (Version=2, TCOBASE=0x0460)\r\n[ 11.204912] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)\r\n[\x1b[0;32m OK \x1b[0m] Found device /dev/vg_nvme/lv_5.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Monitoring of LVM2 mirrors,\xe2\x80\xa6sing dmeventd or progress polling.\r\r\n[\x1b[0;32m OK \x1b[0m] Started LVM event activation on device 259:0.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\r\n Starting Tell Plymouth To Write Out Runtime Data...\r\r\n Starting Restore /run/initramfs on shutdown...\r\r\n Starting Import network configuration from initramfs...\r\r\n Mounting /var/lib/ceph...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Restore /run/initramfs on shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Tell Plymouth To Write Out Runtime Data.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Import network configuration from initramfs.\r\r\n Starting Create Volatile Files and Directories...\r\r\n[ 11.477533] SGI XFS with ACLs, security attributes, no debug enabled\r\n[ 11.485872] XFS (dm-4): Mounting V4 Filesystem\r\n[ 11.494939] XFS (dm-4): Ending clean mount\r\n[\x1b[0;32m OK \x1b[0m] Mounted /var/lib/ceph.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directories.\r\r\n Starting Security Auditing Service...\r\r\n Mounting RPC Pipe File System...\r\r\n Starting RPC Bind...\r\r\n[ 11.576184] RPC: Registered named UNIX socket transport module.\r\n[ 11.582627] RPC: Registered udp transport module.\r\n[ 11.587799] RPC: Registered tcp transport module.\r\n[ 11.593160] RPC: Registered tcp NFSv4.1 backchannel transport module.\r\n[\x1b[0;32m OK \x1b[0m] Mounted RPC Pipe File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target rpc_pipefs.target.\r\r\n[\x1b[0;32m OK \x1b[0m] Started RPC Bind.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Security Auditing Service.\r\r\n Starting Update UTMP about System Boot/Shutdown...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Update UTMP about System Boot/Shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Started daily update of the root trust anchor for DNSSEC.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Generate summary of yesterday\'s process accounting.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Daily Cleanup of Temporary Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on SSSD Kerberos Cache Manager responder socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Run system activity accounting tool every 10 minutes.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on D-Bus System Message Bus Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Basic System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started D-Bus System Message Bus.\r\r\n Starting System Security Services Daemon...\r\r\n Starting Kernel Samepage Merging...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Machine Check Exception Logging Daemon.\r\r\n Starting NTP client/server...\r\r\n Starting Load CPU microcode update...\r\r\n Starting Resets System Activity Logs...\r\r\n Starting Self Monitoring and Reporting Technology (SMART) Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Ceph crash dump collector.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6 ceph*@.service instances at once.\r\r\n Starting GSSAPI Proxy Daemon...\r\r\n Starting Network Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Started irqbalance daemon.\r\r\n Starting VDO volume services...\r\r\n[\x1b[0;32m OK \x1b[0m] Started libstoragemgmt plug-in server daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target sshd-keygen.target.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging.\r\r\n Starting Kernel Samepage Merging (KSM) Tuning Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load CPU microcode update.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Resets System Activity Logs.\r\r\n[\x1b[0;32m OK \x1b[0m] Started GSSAPI Proxy Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging (KSM) Tuning Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target NFS client services.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started NTP client/server.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Self Monitoring and Reporting Technology (SMART) Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Network.\r\r\n Starting OpenSSH server daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Nagios Remote Plugin Executor.\r\r\n Starting /etc/rc.d/rc.local Compatibility...\r\r\n+ \'[\' \'!\' -f /.cephlab_net_configured \']\'\r\n+ set +e\r\n+ attempts=0\r\n+ myips=\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 0 -ge 10 \']\'\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ ip -4 addr\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n Starting Network Manager Wait Online...\r\r\n Starting Hostname Service...\r\r\n+ myips=\r\n+ attempts=1\r\n+ sleep 1\r\n[\x1b[0;32m OK \x1b[0m] Started OpenSSH server daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hostname Service.\r\r\n Starting Network Manager Script Dispatcher Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager Script Dispatcher Service.\r\r\n[ 13.917555] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.975548] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.984082] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[ 14.036709] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[ 14.045217] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 14.088901] ixgbe 0000:03:00.0: registered PHC device on enp3s0f0\r\n[\x1b[0;32m OK \x1b[0m] Started System Security Services Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target User and Group Name Lookups.\r\r\n Starting Login Service...\r\r\n Starting Permit User Sessions...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Permit User Sessions.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Job spooling tools.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Command Scheduler.\r\r\n[\x1b[0;32m OK \x1b[0m] Started VDO volume services.\r\r\n[ 14.199889] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 14.209011] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n[ 14.266886] ixgbe 0000:03:00.1: registered PHC device on enp3s0f1\r\n[\x1b[0;32m OK \x1b[0m] Started Login Service.\r\r\n[ 14.375925] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n[ 14.468277] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 1 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=2\r\n+ sleep 1\r\n[ 14.591131] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX\r\n[ 14.599320] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0f1: link becomes ready\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 2 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=3\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 3 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=4\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 4 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=172.21.15.122\r\n+ attempts=5\r\n+ sleep 1\r\n+ \'[\' 172.21.15.122 \'!=\' \'\' \']\'\r\n+ set -e\r\n+ \'[\' -n 172.21.15.122 \']\'\r\n+ for ip in $myips\r\n+ timeout 1s ping -I 172.21.15.122 -nq -c1 172.21.0.1\r\n++ dig +short -x 172.21.15.122 @172.21.0.1\r\n++ sed \'s/\\.com.*/\\.com/g\'\r\n+ newhostname=smithi122.front.sepia.ceph.com\r\n+ \'[\' -n smithi122.front.sepia.ceph.com \']\'\r\n+ hostname smithi122.front.sepia.ceph.com\r\n++ hostname -d\r\n+ newdomain=front.sepia.ceph.com\r\n++ hostname -s\r\n+ shorthostname=smithi122\r\n+ echo smithi122\r\n+ grep -q front.sepia.ceph.com /etc/hosts\r\n+ sed -i \'s/.*front.sepia.ceph.com.*/172.21.15.122 smithi122.front.sepia.ceph.com smithi122/g\' /etc/hosts\r\n+ break\r\n+ command -v zypper\r\n+ command -v apt-get\r\n+ \'[\' -e /.cephlab_rc_local \']\'\r\n+ exit 0\r\n[\x1b[0;32m OK \x1b[0m] Started /etc/rc.d/rc.local Compatibility.\r\r\n Starting Terminate Plymouth Boot Screen...\r\r\n Starting Hold until boot process finishes up...\r\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n' 2020-11-16T22:29:49.278 DEBUG:teuthology.orchestra.console:expect after: b'smithi122 login: ' 2020-11-16T22:29:49.279 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:49.280 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:49.280 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:49.293+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:49.280 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:49.380 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:49.394+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:49.425 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:49.425 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:49.430 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:29:49.431 INFO:teuthology.misc:Re-opening connections... 2020-11-16T22:29:49.431 INFO:teuthology.misc:trying to connect to ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:29:49.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:29:49.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:29:49.436 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:49.481 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:49.495+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:49.580 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:49.581 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:49.583 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:49.597+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:49.591 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:49.638 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:49.684 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:49.698+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:49.774 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:49.783 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:49.804 INFO:teuthology.orchestra.run.smithi122:> true 2020-11-16T22:29:49.804 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:49.805 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:49.805 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:49.805 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:49.805 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:49.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:49.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:49.818 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:49.832+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:49.886 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:49.900+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:49.905 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:49.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:49.905 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:29:49.905 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:49.914 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:49.971 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:29:49.998 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:50.013+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:50.087 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:50.101+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:50.087 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:50.087 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:29:50.088 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:29:50.088 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:29:50.088 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:50.097 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:50.187 DEBUG:teuthology.misc:waited 0.7554540634155273 2020-11-16T22:29:50.189 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:50.203+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:50.265 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:29:50.265 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:29:50.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 392.77024030299998, 2020-11-16T22:29:50.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:29:50.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:29:50.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:29:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:29:50.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 392.77012366000002, 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.278 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:29:50.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:29:50.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:50.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.288 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 392.76997529099998, 2020-11-16T22:29:50.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:29:50.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:29:50.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:29:50.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:29:50.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.300 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 392.76919186200001, 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:29:50.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:29:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:29:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:29:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.310 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:2002 4.7 4:f3dfad11:::100000009fd.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected e87)", 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 96.213155434000001, 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001, 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:50.311 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 2002 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:50.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065409+0000", 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8474000000000003e-05 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065467+0000", 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065476+0000", 2020-11-16T22:29:50.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5943e-05 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065492+0000", 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2620000000000004e-06 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065499+0000", 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5781999999999997e-05 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065534+0000", 2020-11-16T22:29:50.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117157 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065652+0000", 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000124065 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.065776+0000", 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066859700000000005 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.066444+0000", 2020-11-16T22:29:50.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0080184169999999999 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074463+0000", 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7459000000000005e-05 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074540+0000", 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137651 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.074678+0000", 2020-11-16T22:29:50.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092690120000000001 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:50.321 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 95.500885944000004, 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:29:50.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:29:50.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.329 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.530 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:29:50.530 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 94.024459249000003, 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:29:50.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:29:50.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:29:50.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.538 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.954236784000003, 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:50.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:50.540 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:29:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:29:50.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:29:50.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:50.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:29:50.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:29:50.954 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:50.954 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.304301053000003, 2020-11-16T22:29:50.954 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:50.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.956 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:29:50.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:29:50.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:29:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:50.962 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.298553745000007, 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:50.963 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:29:50.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:50.964 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:51.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:51.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:29:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:29:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:51.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:29:51.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:51.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.378 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.297417741000004, 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:51.379 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:51.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:51.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:51.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:29:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:29:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:51.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:29:51.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:29:51.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:51.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:29:51.591 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.296270645000007, 2020-11-16T22:29:51.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:29:51.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:29:51.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:29:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:29:51.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:29:51.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:51.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:29:51.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:29:51.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:29:51.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.290329272999998, 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:51.805 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:51.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:29:51.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:52.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:29:52.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:52.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:29:52.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:52.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:29:52.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:52.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:29:52.017 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.287984534000003, 2020-11-16T22:29:52.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:29:52.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.020 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:52.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:29:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:29:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:29:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:29:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.283238397000005, 2020-11-16T22:29:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:29:52.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:52.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:52.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:29:52.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:29:52.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:29:52.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:52.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:52.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:29:52.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:29:52.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.646 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:52.646 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 92.764371854000004, 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:29:52.647 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:52.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:52.649 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:29:52.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.651 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:29:52.652 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.653 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:29:52.654 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:52.655 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 92.245355087999997, 2020-11-16T22:29:52.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:29:52.656 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:52.656 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:52.656 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:52.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:52.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:52.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:29:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:29:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:52.867 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:53.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:29:53.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:29:53.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:29:53.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:53.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 91.068547918999997, 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:53.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:29:53.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:53.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:29:53.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:53.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:29:53.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:53.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:29:53.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:29:53.282 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.282 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:53.282 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 91.051855716999995, 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:53.283 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:29:53.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:53.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:29:53.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:29:53.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:53.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:29:53.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:29:53.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.493 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 90.022670141000006, 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:29:53.494 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:29:53.495 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:53.698 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:29:53.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:29:53.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:53.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:29:53.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:29:53.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:29:53.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:29:53.705 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:29:53.706 DEBUG:tasks.ceph_manager:Mounting data for osd.2 on ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:29:53.706 INFO:tasks.ceph_manager:Mounting osd.2: dev: ubuntu@smithi122.front.sepia.ceph.com, cluster: cephmountpoint: /var/lib/ceph/osd/ceph-2, type: xfs, options: ['noatime'] 2020-11-16T22:29:53.706 INFO:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2020-11-16T22:29:53.707 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:29:53.708 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:29:53.709 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:53.720+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:53.710 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:53.807 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:53.821+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:53.855 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:53.855 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:53.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:27148 4.3 4:cd35a60f:::10000001887.00000000:head [zero 59516~49487] snapc 1=[] ondisk+write+known_if_redirected e106)", 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:53.869689+0000", 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00068989599999999996, 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00070711899999999998, 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:29:53.856 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 27148 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:53.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869689+0000", 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869689+0000", 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9692999999999999e-05 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869709+0000", 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8670000000000004e-06 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869713+0000", 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3380000000000005e-06 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869721+0000", 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2200000000000001e-06 2020-11-16T22:29:53.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:54.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:54.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:54.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869724+0000", 2020-11-16T22:29:54.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5374999999999998e-05 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869780+0000", 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0973000000000001e-05 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:54.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:53.869841+0000", 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000151466 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:54.064 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:29:54.065 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:29:54.065 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:54.067 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:29:54.067 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:29:54.067 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:54.067 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:54.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:54.068 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:54.068 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:54.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.079+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.069 INFO:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:29:54.111 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T22:29:54.111 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T22:29:54.153 INFO:tasks.ceph.osd.2:Started 2020-11-16T22:29:54.153 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:29:54.355 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:54.355 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:54.356 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:54.356 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.455 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.469+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.537 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:54.538 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:54.548 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:54.591 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:54.605+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:54.657 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:54.671+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:54.681 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:54.681 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:54.690 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:54.758 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.771+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~157s 2020-11-16T22:29:54.858 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.872+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.868 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:54.868 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:54.878 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:54.959 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:54.972+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:29:55.006 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.007 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2020-11-16T22:29:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:0 2020-11-16T22:29:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:1 2020-11-16T22:29:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2 2020-11-16T22:29:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2020-11-16T22:29:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2020-11-16T22:29:55.010 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-16T22:29:55.010 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2020-11-16T22:29:55.016 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:55.060 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.074+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:55.152 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.153 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.161 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.175+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:55.164 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:55.262 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.277+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:55.305 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.306 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.316 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:55.362 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.377+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:55.452 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.452 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.462 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:29:55.464 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.478+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:55.525 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:29:55.528+0000 7f5a24f6fec0 -1 Falling back to public interface 2020-11-16T22:29:55.564 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:55.578+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:55.597 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.597 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:29:55.665 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:55.678+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:55.734 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.734 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.743 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:29:55.766 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:55.779+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:55.867 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:55.880+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:55.874 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:55.875 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:55.884 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:29:55.968 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:55.982+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:56.030 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:56.030 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:56.040 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:29:56.068 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:56.082+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:56.069 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:29:56.169 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:29:56.183+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:29:56.182 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:29:56.182 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:29:56.192 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:29:56.215 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:29:56.236 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:29:56.236 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:29:56.236 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:29:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.2 2020-11-16T22:29:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: problem getting command descriptions from osd.2 2020-11-16T22:29:56.239 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:29:56.239 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:29:56.270 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:29:56.283+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:29:56.343 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:29:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:29:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:29:56.343 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 111.035710776, 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:56.344 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:56.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:29:56.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:29:56.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:56.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:29:56.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:29:56.353 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:56.354 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 106.035207888, 2020-11-16T22:29:56.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:29:56.354 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:56.354 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:56.354 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:29:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:29:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:29:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:29:56.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:56.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:29:56.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:29:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:29:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:56.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:29:56.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:29:56.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:29:56.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:56.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 100.106536656, 2020-11-16T22:29:56.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:29:56.771 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:56.771 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:56.771 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:56.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:29:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:29:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:29:57.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:29:57.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:57.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:29:57.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:57.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.186 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:57.187 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:57.187 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 99.389250184999995, 2020-11-16T22:29:57.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:29:57.187 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:57.390 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:57.390 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:57.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:57.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:29:57.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:57.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:29:57.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:57.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:29:57.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:29:57.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:57.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:29:57.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:29:57.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:57.602 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:57.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.603 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.145087971999999, 2020-11-16T22:29:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:29:57.806 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:57.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:57.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:29:57.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:29:57.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:29:58.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:29:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:58.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:29:58.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:29:58.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:58.222 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:58.222 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:58.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1104 3.6 3:7bf7e7b5:::200.00000006:head [write 1365546~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.144436131000006, 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987, 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:58.223 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211851+0000", 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7029000000000001e-05 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:58.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211888+0000", 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0530000000000001e-06 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211895+0000", 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0930000000000003e-06 2020-11-16T22:29:58.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:58.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211903+0000", 2020-11-16T22:29:58.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8439999999999998e-06 2020-11-16T22:29:58.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:58.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211906+0000", 2020-11-16T22:29:58.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.018756096999999999 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230662+0000", 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0099000000000002e-05 2020-11-16T22:29:58.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230722+0000", 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010809 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.230830+0000", 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033763450000000002 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234207+0000", 2020-11-16T22:29:58.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041465110000000003 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238353+0000", 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4829000000000001e-05 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:58.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238398+0000", 2020-11-16T22:29:58.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5996999999999997e-05 2020-11-16T22:29:58.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:58.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238484+0000", 2020-11-16T22:29:58.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026632987 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.639 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1105 3.6 3:7bf7e7b5:::200.00000006:head [write 1366651~1383 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.144353816999995, 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999, 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:58.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:58.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211933+0000", 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7043999999999999e-05 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211980+0000", 2020-11-16T22:29:58.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211986+0000", 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:58.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211995+0000", 2020-11-16T22:29:58.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3890000000000001e-06 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211999+0000", 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019211281 2020-11-16T22:29:58.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231210+0000", 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0940000000000003e-05 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231271+0000", 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4691000000000005e-05 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231366+0000", 2020-11-16T22:29:58.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028528939999999999 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234219+0000", 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043055439999999997 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238524+0000", 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3726000000000001e-05 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:58.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:59.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238568+0000", 2020-11-16T22:29:59.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8324999999999994e-05 2020-11-16T22:29:59.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238636+0000", 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026702700999999999 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:59.055 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.144261188000002, 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:29:59.056 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:59.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:59.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:29:59.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:29:59.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:59.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:29:59.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:29:59.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:59.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:29:59.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:59.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:29:59.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:29:59.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:29:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:29:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.1441789, 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:29:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:29:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:29:59.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:29:59.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:29:59.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:29:59.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:29:59.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:29:59.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:29:59.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:29:59.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:29:59.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:29:59.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:29:59.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:29:59.888 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:59.889 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.144096555999994, 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:29:59.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:29:59.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:29:59.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:29:59.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:29:59.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:29:59.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:00.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:00.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:00.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:00.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:00.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:00.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:00.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:00.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:00.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:00.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:00.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.144021436000003, 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:00.306 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:00.307 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:00.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:00.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:00.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:00.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:00.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:00.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:00.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:00.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:00.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:00.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:00.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:00.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:00.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.722 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:00.723 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:00.723 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143942502000002, 2020-11-16T22:30:00.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:00.723 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:00.723 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:00.926 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:00.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:00.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:00.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:00.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:00.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:00.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:00.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:00.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:01.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:01.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:01.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:01.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:01.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:01.138 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:01.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.139 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143865079999998, 2020-11-16T22:30:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:01.342 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:01.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:01.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:01.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:01.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:01.552 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.553 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:01.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:01.555 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:01.556 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143784917999994, 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:01.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:01.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:01.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:01.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:01.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:01.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:01.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:01.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:01.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:01.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:01.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:01.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:01.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:01.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:01.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:01.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:01.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:01.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:01.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:01.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:02.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:02.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:02.174 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:02.174 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:02.175 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:02.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.175 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:02.175 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:02.176 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143703105, 2020-11-16T22:30:02.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:02.176 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:02.176 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:02.176 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:02.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:02.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:02.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:02.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:02.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:02.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:02.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:02.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:02.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:02.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:02.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:02.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:02.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:02.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:02.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:02.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:02.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:02.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:02.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:02.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:02.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:02.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:02.591 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:02.592 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.592 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.592 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:02.593 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:02.593 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143609718999997, 2020-11-16T22:30:02.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:02.594 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:02.594 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:02.594 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:02.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:02.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:02.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:02.595 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:02.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:02.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:02.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:02.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:02.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:02.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:02.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:02.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:02.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:03.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:03.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:03.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143489322999997, 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.143389256000006, 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:03.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:03.631 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:03.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:03.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:03.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:03.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:03.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 55.743116272000002, 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:04.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:04.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:04.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:04.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:04.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:04.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:04.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:04.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:04.258 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:04.259 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.897143338999999, 2020-11-16T22:30:04.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:04.259 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:04.259 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:04.259 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:04.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:04.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:04.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:04.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:04.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:04.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:04.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:04.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:04.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:04.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:04.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:04.673 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:04.674 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:30:04.675 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:04.676 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:04.678 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:29:57.092+0000 7f5a24f6fec0 -1 osd.2 69 log_to_monitors {default=true} 2020-11-16T22:30:04.678 INFO:tasks.workunit.client.0.smithi138.stdout:50000 trunc from 0xa4fa5 to 0x60d70 2020-11-16T22:30:04.678 INFO:tasks.workunit.client.0.smithi138.stdout:All 50000 operations completed A-OK! 2020-11-16T22:30:04.679 INFO:tasks.workunit.client.0.smithi138.stdout:Seed set to 1 2020-11-16T22:30:04.679 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x5a0dfb 2020-11-16T22:30:04.679 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x5c415c 2020-11-16T22:30:04.679 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x9c27b4 2020-11-16T22:30:04.679 INFO:tasks.workunit.client.0.smithi138.stdout:fallocating to largest ever: 0xa00000 2020-11-16T22:30:04.882 INFO:tasks.workunit.client.0.smithi138.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2020-11-16T22:30:04.882 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2020-11-16T22:30:04.882 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2020-11-16T22:30:04.882 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support clone range, disabling! 2020-11-16T22:30:04.883 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support dedupe range, disabling! 2020-11-16T22:30:04.883 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:04.689+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:04.883 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:04.886 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:04.899+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:04.886 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:04.886 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:04.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:04.886 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17397 3.0 3:1e8917d7:::200.0000000e:head [write 272678~1926 [fadvise_dontneed] in=1926b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force e111)", 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.8214818689999999, 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8215021149999999, 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "delayed", 2020-11-16T22:30:04.887 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:04.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7129999999999999e-05 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:04.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008048+0000", 2020-11-16T22:30:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3217999999999999e-05 2020-11-16T22:30:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008061+0000", 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3074000000000001e-05 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008074+0000", 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5650000000000001e-06 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008080+0000", 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5339000000000002e-05 2020-11-16T22:30:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008115+0000", 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0489999999999998e-06 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008122+0000", 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007381182 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.015504+0000", 2020-11-16T22:30:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5389999999999996e-06 2020-11-16T22:30:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.015513+0000", 2020-11-16T22:30:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.012030357 2020-11-16T22:30:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:02.027543+0000", 2020-11-16T22:30:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8615999999999998e-05 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for readable", 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:02.027572+0000", 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0195610690000001 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17398 3.0 3:1e8917d7:::200.0000000e:head [write 274604~7012 [fadvise_dontneed] in=7012b] snapc 0=[] ondisk+write+known_if_redirected+full_force e114)", 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.62136229200000004, 2020-11-16T22:30:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.62143658999999996, 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "delayed", 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6254000000000001e-05 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208177+0000", 2020-11-16T22:30:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2215000000000001e-05 2020-11-16T22:30:05.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208189+0000", 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2486999999999999e-05 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208201+0000", 2020-11-16T22:30:05.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1840000000000004e-06 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208208+0000", 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7455999999999997e-05 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208295+0000", 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3952000000000003e-05 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for readable", 2020-11-16T22:30:05.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208329+0000", 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198548 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:05.510 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:30:05.511 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:05.511 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2020-11-16T22:30:05.514 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:05.527+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:05.514 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:05.514 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:05.514 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:05.515 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:05.515 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:05.515 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:05.515 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:05.515 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:05.516 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:05.516 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:05.516 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:05.613 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:05.627+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:05.683 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:05.683 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:05.692 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:05.737 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:05.751+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:05.814 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:05.828+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:05.817 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:05.817 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:05.826 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:05.914 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:05.929+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:05.964 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:05.965 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:05.974 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:06.015 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:06.030+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:06.031 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:30:06.031 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2020-11-16T22:30:06.044 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2020-11-16T22:30:06.116 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:06.130+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:06.155 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.221 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2020-11-16T22:30:06.223 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:06.223 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:06.236+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:06.322 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:06.337+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:06.353 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.354 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.364 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:06.424 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:06.438+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:06.504 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.505 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.514 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:06.524 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:06.539+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:06.627 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:06.641+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:06.648 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.649 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.659 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:06.729 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:06.743+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.800 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.811 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:06.830 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:06.844+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:06.931 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:06.945+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:06.948 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:06.949 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:06.958 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:07.033 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:07.046+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:07.034 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2020-11-16T22:30:07.046 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2020-11-16T22:30:07.103 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:07.103 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:07.115 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:07.133 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:07.147+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:07.233 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:07.248+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:07.245 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:07.245 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:07.254 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:07.334 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:07.348+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:07.391 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:07.400 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:07.434 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:07.448+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:07.516 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:07.535 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:07.550+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:07.540 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:07.540 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:07.550 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:07.635 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:07.650+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:07.656 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:07.676 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:07.697 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:07.698 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:07.699 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:07.699 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:07.723 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:07.723 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:30:07.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:30:07.723 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.415758608, 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.724 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:30:07.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:30:07.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:30:07.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 117.41525572, 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.734 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:30:07.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:30:07.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 111.48658448800001, 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.744 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:30:07.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:30:07.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:30:07.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:30:07.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.754 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 110.769298017, 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:07.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:30:07.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:30:07.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.525135804, 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.765 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:30:07.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:30:07.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:30:07.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.775 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.52430902, 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:30:07.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:30:07.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:30:07.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.524226732, 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.786 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:30:07.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:30:07.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:30:07.795 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:07.796 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.524144388, 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:07.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:07.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:07.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.52406926800001, 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.807 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:07.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:07.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:07.816 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.523990334, 2020-11-16T22:30:07.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:07.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:07.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:07.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.523912912, 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:07.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:07.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:07.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:07.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.837 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.52383275, 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:07.838 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:07.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:07.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:07.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:07.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.848 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.523750937, 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:07.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:07.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:07.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:07.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.523657551, 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.859 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:07.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:07.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:07.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.869 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.523537155, 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:07.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:07.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:07.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:07.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.52343708799999, 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.880 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:07.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:07.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:07.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:07.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:07.890 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 67.123164103999997, 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:07.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:07.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:07.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 31.277191170999998, 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.901 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:07.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:07.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:07.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 8.8263890299999996, 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.910 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:30:07.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:30:07.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 8.5280119709999997, 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:07.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:07.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:30:07.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:30:07.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:30:07.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:30:07.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:30:07.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:07.931 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:07.932 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:07.933 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:07.947+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:08.032 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:08.047+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:08.041 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2020-11-16T22:30:08.044 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:30:08.054 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-11-16T22:30:08.054 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17397 3.0 3:1e8917d7:::200.0000000e:head [write 272678~1926 [fadvise_dontneed] in=1926b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force e111)", 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0826676610000003, 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0826818879999998, 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "delayed", 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008011+0000", 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7129999999999999e-05 2020-11-16T22:30:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008048+0000", 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3217999999999999e-05 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008061+0000", 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3074000000000001e-05 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008074+0000", 2020-11-16T22:30:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5650000000000001e-06 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008080+0000", 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5339000000000002e-05 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008115+0000", 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0489999999999998e-06 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:30:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.008122+0000", 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007381182 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.015504+0000", 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5389999999999996e-06 2020-11-16T22:30:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:01.015513+0000", 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.012030357 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:02.027543+0000", 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8615999999999998e-05 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for readable", 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:02.027572+0000", 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0195610690000001 2020-11-16T22:30:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17398 3.0 3:1e8917d7:::200.0000000e:head [write 274604~7012 [fadvise_dontneed] in=7012b] snapc 0=[] ondisk+write+known_if_redirected+full_force e114)", 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.8825480840000002, 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8826147209999999, 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "delayed", 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208131+0000", 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6254000000000001e-05 2020-11-16T22:30:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208177+0000", 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2215000000000001e-05 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208189+0000", 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2486999999999999e-05 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208201+0000", 2020-11-16T22:30:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1840000000000004e-06 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208208+0000", 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7455999999999997e-05 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208295+0000", 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3952000000000003e-05 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for readable", 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:04.208329+0000", 2020-11-16T22:30:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000198548 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:30:08.093 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:08.094 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:08.133 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:08.147+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:08.226 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:30:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:30:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:30:08.227 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:08.234 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:08.248+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:08.235 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:08.335 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:08.349+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:08.374 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:30:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:30:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.387081801000001, 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:30:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:30:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:30:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:30:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.1789316339999996, 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:30:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:30:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:30:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:30:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:30:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:30:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.358581203, 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001, 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025738089999999998 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031142+0000", 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.242e-05 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031155+0000", 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6178e-05 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031171+0000", 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1250000000000004e-06 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031178+0000", 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5498000000000002e-05 2020-11-16T22:30:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031243+0000", 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1666e-05 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031255+0000", 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099647690000000001 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.041220+0000", 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001 2020-11-16T22:30:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3371261329999999, 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293, 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010396590000000001 2020-11-16T22:30:08.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051063+0000", 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1077000000000002e-05 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051084+0000", 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6332000000000001e-05 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051111+0000", 2020-11-16T22:30:08.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2245e-05 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051123+0000", 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000182122 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051305+0000", 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2546e-05 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051318+0000", 2020-11-16T22:30:08.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011890312 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.063208+0000", 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.414 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.061206+0000", 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3259434120000009, 2020-11-16T22:30:08.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010681424, 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.061206+0000", 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.061206+0000", 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001012864 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062219+0000", 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2659000000000001e-05 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062242+0000", 2020-11-16T22:30:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0120999999999998e-05 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062272+0000", 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7869999999999996e-06 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062282+0000", 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8641999999999997e-05 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062360+0000", 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.377e-05 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.062374+0000", 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095135810000000001 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.071888+0000", 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010681424 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3194981729999995, 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927, 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001385074 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069037+0000", 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9641999999999999e-05 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069056+0000", 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0559000000000001e-05 2020-11-16T22:30:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069077+0000", 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0624e-05 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069087+0000", 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0131999999999998e-05 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069168+0000", 2020-11-16T22:30:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3397000000000001e-05 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069181+0000", 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099632720000000004 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079144+0000", 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3095948960000001, 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001, 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012733930000000001 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078828+0000", 2020-11-16T22:30:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2857000000000002e-05 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078851+0000", 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9649e-05 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078881+0000", 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4674999999999999e-05 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078895+0000", 2020-11-16T22:30:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9827000000000006e-05 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078985+0000", 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4489e-05 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079000+0000", 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010194923 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.089195+0000", 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.438 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3032205510000008, 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:08.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:30:08.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bf8dcf43:::200.00000008:head, version: 93'1034, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bf8dcf43:::200.00000008:head@93'1034, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.098978+0000", 2020-11-16T22:30:08.446 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.2881713270000006, 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0098224279999999994, 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.098978+0000", 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.098978+0000", 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002448315 2020-11-16T22:30:08.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101427+0000", 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.262e-05 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101449+0000", 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7724000000000001e-05 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101477+0000", 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.226e-05 2020-11-16T22:30:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101489+0000", 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1743999999999997e-05 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101551+0000", 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4929999999999998e-06 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.101559+0000", 2020-11-16T22:30:08.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007241272 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.108801+0000", 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0098224279999999994 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bf8dcf43:::200.00000008:head, version: 93'1034, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bf8dcf43:::200.00000008:head@93'1034, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.115824+0000", 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.2713258970000005, 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091559269999999995, 2020-11-16T22:30:08.454 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.115824+0000", 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.115824+0000", 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010730169999999999 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.116897+0000", 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1771e-05 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.116909+0000", 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6569e-05 2020-11-16T22:30:08.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.116925+0000", 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.116931+0000", 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4510000000000004e-05 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.116995+0000", 2020-11-16T22:30:08.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0715e-05 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.117006+0000", 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079738159999999999 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.124980+0000", 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091559269999999995 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.4 110/109 [PushOp(3:2e2fa760:::200.00000001:head, version: 87'57, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:2e2fa760:::200.00000001:head@87'57, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.132088+0000", 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.2550618280000005, 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080631729999999999, 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.462 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.132088+0000", 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.132088+0000", 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001472288 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133560+0000", 2020-11-16T22:30:08.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1591e-05 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133572+0000", 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4252e-05 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133586+0000", 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6370000000000005e-06 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133595+0000", 2020-11-16T22:30:08.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5151e-05 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133660+0000", 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0259e-05 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.133670+0000", 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064809949999999998 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.140151+0000", 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080631729999999999 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.469 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.2 110/109 [PushOp(3:5e41ab69:::200.00000005:head, version: 88'7, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:5e41ab69:::200.00000005:head@88'7, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.187252+0000", 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1998973389999996, 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087874189999999994, 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.187252+0000", 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.187252+0000", 2020-11-16T22:30:08.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082876900000000003 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188081+0000", 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188091+0000", 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6677e-05 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188108+0000", 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1910000000000003e-06 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188115+0000", 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4975999999999995e-05 2020-11-16T22:30:08.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188190+0000", 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.579e-06 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.188196+0000", 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0078434349999999993 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.196040+0000", 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087874189999999994 2020-11-16T22:30:08.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.6 110/109 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.189334+0000", 2020-11-16T22:30:08.477 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1978155620000006, 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084016070000000002, 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.189334+0000", 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.189334+0000", 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095577800000000001 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190290+0000", 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0842999999999999e-05 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190301+0000", 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6418000000000001e-05 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190317+0000", 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0380000000000002e-06 2020-11-16T22:30:08.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190324+0000", 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3466999999999994e-05 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190388+0000", 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0692e-05 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.190398+0000", 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073373709999999997 2020-11-16T22:30:08.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.197736+0000", 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084016070000000002 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:17287 3.0 e110/109 3:1e8917d7:::200.0000000e:head v 110'4394, mlcod=95'2154)", 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.276324+0000", 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1108252459999992, 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0082270940000000008, 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.485 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276324+0000", 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276324+0000", 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5104000000000001e-05 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276360+0000", 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4389999999999999e-06 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276365+0000", 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3159999999999997e-06 2020-11-16T22:30:08.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276373+0000", 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5079999999999999e-06 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.276378+0000", 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050574230000000001 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.281435+0000", 2020-11-16T22:30:08.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9267999999999999e-05 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.281454+0000", 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030599619999999998 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.284514+0000", 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7073999999999998e-05 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.284552+0000", 2020-11-16T22:30:08.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0082270940000000008 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:17291 3.0 e110/109 3:1e8917d7:::200.0000000e:head v 110'4395, mlcod=95'2154)", 2020-11-16T22:30:08.493 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.298509+0000", 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.0886401750000001, 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089179340000000006, 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298509+0000", 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298509+0000", 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3366000000000001e-05 2020-11-16T22:30:08.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298593+0000", 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9019999999999999e-06 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298601+0000", 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3611000000000001e-05 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298614+0000", 2020-11-16T22:30:08.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1210000000000003e-06 2020-11-16T22:30:08.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.298620+0000", 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062931580000000001 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.304914+0000", 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7844e-05 2020-11-16T22:30:08.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.304941+0000", 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024413529999999998 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:08.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.307383+0000", 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.307427+0000", 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089179340000000006 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.705 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.3 110/109 [PushOp(3:d5c7a900:::200.00000003:head, version: 88'22, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:d5c7a900:::200.00000003:head@88'22, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.449964+0000", 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.9371856320000003, 2020-11-16T22:30:08.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094737599999999995, 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.449964+0000", 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.449964+0000", 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093784000000000001 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.450902+0000", 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0311e-05 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.450912+0000", 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7008999999999999e-05 2020-11-16T22:30:08.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.450929+0000", 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.165e-06 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.450936+0000", 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5075999999999998e-05 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.451001+0000", 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0908e-05 2020-11-16T22:30:08.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.451012+0000", 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084254510000000005 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.459438+0000", 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094737599999999995 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.3 110/109 [PushOp(3:ceffbd45:::200.0000000a:head, version: 97'2197, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ceffbd45:::200.0000000a:head@97'2197, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.713 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.473701+0000", 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.9134481790000004, 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088324909999999996, 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.473701+0000", 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.473701+0000", 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095124300000000005 2020-11-16T22:30:08.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474653+0000", 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1443999999999999e-05 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474664+0000", 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6974e-05 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474681+0000", 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7920000000000004e-06 2020-11-16T22:30:08.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474688+0000", 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1442000000000001e-05 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474729+0000", 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.686e-06 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.474736+0000", 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077979099999999999 2020-11-16T22:30:08.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.482534+0000", 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088324909999999996 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.3 110/109 [PushOp(3:ceffbd45:::200.0000000a:head, version: 97'2197, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ceffbd45:::200.0000000a:head@97'2197, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.496195+0000", 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.8909543959999997, 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085482579999999996, 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.496195+0000", 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.496195+0000", 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094319800000000002 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497138+0000", 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1938e-05 2020-11-16T22:30:08.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497150+0000", 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7303999999999999e-05 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497168+0000", 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4279999999999999e-06 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497175+0000", 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3655e-05 2020-11-16T22:30:08.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497229+0000", 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8220000000000007e-06 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.497237+0000", 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075069129999999996 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.504744+0000", 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085482579999999996 2020-11-16T22:30:08.727 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.432565587, 2020-11-16T22:30:08.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:30:08.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:30:08.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.735 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.2 117/109 [PushOp(4:584e1577:::10000000217.00000001:head, version: 86'1, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:584e1577:::10000000217.00000001:head@86'1, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.39437690199999997, 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001, 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:08.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093718100000000004 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993710+0000", 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3156000000000001e-05 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993723+0000", 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8623000000000001e-05 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993742+0000", 2020-11-16T22:30:08.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993749+0000", 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9206e-05 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993829+0000", 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2907e-05 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993842+0000", 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010134260000000001 2020-11-16T22:30:08.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:08.003976+0000", 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:08.743 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:08.743 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:08.745 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:08.746 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2020-11-16T22:30:08.050556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86786188,"num_objects":3596,"num_object_clones":0,"num_object_copies":7192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":14490,"num_read_kb":365202,"num_write":36235,"num_write_kb":377269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":58589380,"num_keys_recovered":84,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29478,"ondisk_log_size":29478,"up":50,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":342444,"kb_used_data":121236,"kb_used_omap":26,"kb_used_meta":221157,"kb_avail":282773076,"statfs":{"total":289910292480,"available":289559629824,"internally_reserved":0,"allocated":124145664,"data_stored":153529278,"data_compressed":14725255,"data_compressed_allocated":50429952,"data_compressed_original":100847310,"omap_allocated":27325,"internal_metadata":226465091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1242219,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":6768,"num_write":138,"num_write_kb":2324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":3696658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018447"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"1079","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029393+0000","last_change":"2020-11-16T22:30:00.040022+0000","last_active":"2020-11-16T22:30:03.029393+0000","last_peered":"2020-11-16T22:30:03.029393+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.035488+0000","last_became_peered":"2020-11-16T22:30:00.035488+0000","last_unstale":"2020-11-16T22:30:03.029393+0000","last_undegraded":"2020-11-16T22:29:59.951911+0000","last_fullsized":"2020-11-16T22:29:59.934996+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":455}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"109","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024965+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:30:03.024965+0000","last_peered":"2020-11-16T22:30:03.024965+0000","last_clean":"2020-11-16T22:30:03.024965+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:30:03.024965+0000","last_undegraded":"2020-11-16T22:30:03.024965+0000","last_fullsized":"2020-11-16T22:30:03.024965+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"97'1035","reported_seq":"1098","reported_epoch":"117","state":"remapped+peering","last_fresh":"2020-11-16T22:30:07.044309+0000","last_change":"2020-11-16T22:30:07.044517+0000","last_active":"2020-11-16T22:30:06.044220+0000","last_peered":"2020-11-16T22:30:05.037928+0000","last_clean":"2020-11-16T22:30:05.037928+0000","last_became_active":"2020-11-16T22:30:02.026496+0000","last_became_peered":"2020-11-16T22:30:02.026496+0000","last_unstale":"2020-11-16T22:30:07.044309+0000","last_undegraded":"2020-11-16T22:30:07.044309+0000","last_fullsized":"2020-11-16T22:30:07.044309+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1035,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388768,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1033,"num_write_kb":8391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8388858,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"2355","reported_epoch":"114","state":"active+undersized+remapped","last_fresh":"2020-11-16T22:30:04.313816+0000","last_change":"2020-11-16T22:30:04.314060+0000","last_active":"2020-11-16T22:30:04.313816+0000","last_peered":"2020-11-16T22:30:04.313816+0000","last_clean":"2020-11-16T22:29:54.995271+0000","last_became_active":"2020-11-16T22:29:59.981868+0000","last_became_peered":"2020-11-16T22:29:59.981868+0000","last_unstale":"2020-11-16T22:30:04.313816+0000","last_undegraded":"2020-11-16T22:30:04.313816+0000","last_fullsized":"2020-11-16T22:29:59.911749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":1876427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"1","objects":442}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.043156+0000","last_change":"2020-11-16T22:29:58.917676+0000","last_active":"2020-11-16T22:30:06.043156+0000","last_peered":"2020-11-16T22:30:06.043156+0000","last_clean":"2020-11-16T22:30:06.043156+0000","last_became_active":"2020-11-16T22:29:58.916780+0000","last_became_peered":"2020-11-16T22:29:58.916780+0000","last_unstale":"2020-11-16T22:30:06.043156+0000","last_undegraded":"2020-11-16T22:30:06.043156+0000","last_fullsized":"2020-11-16T22:30:06.043156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4421","reported_epoch":"117","state":"remapped+peering","last_fresh":"2020-11-16T22:30:07.043722+0000","last_change":"2020-11-16T22:30:07.043841+0000","last_active":"2020-11-16T22:30:06.043282+0000","last_peered":"2020-11-16T22:30:05.037854+0000","last_clean":"2020-11-16T22:30:05.037854+0000","last_became_active":"2020-11-16T22:30:02.024620+0000","last_became_peered":"2020-11-16T22:30:02.024620+0000","last_unstale":"2020-11-16T22:30:07.043722+0000","last_undegraded":"2020-11-16T22:30:07.043722+0000","last_fullsized":"2020-11-16T22:30:07.043722+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":16777250,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1111","reported_epoch":"116","state":"active+clean","last_fresh":"2020-11-16T22:30:06.043922+0000","last_change":"2020-11-16T22:30:04.039525+0000","last_active":"2020-11-16T22:30:06.043922+0000","last_peered":"2020-11-16T22:30:06.043922+0000","last_clean":"2020-11-16T22:30:06.043922+0000","last_became_active":"2020-11-16T22:30:04.035094+0000","last_became_peered":"2020-11-16T22:30:04.035094+0000","last_unstale":"2020-11-16T22:30:06.043922+0000","last_undegraded":"2020-11-16T22:30:06.043922+0000","last_fullsized":"2020-11-16T22:30:06.043922+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"109","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024953+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:30:03.024953+0000","last_peered":"2020-11-16T22:30:03.024953+0000","last_clean":"2020-11-16T22:30:03.024953+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:30:03.024953+0000","last_undegraded":"2020-11-16T22:30:03.024953+0000","last_fullsized":"2020-11-16T22:30:03.024953+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"110'4465","reported_seq":"4595","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042894+0000","last_change":"2020-11-16T22:30:02.027789+0000","last_active":"2020-11-16T22:30:06.042894+0000","last_peered":"2020-11-16T22:30:06.042894+0000","last_clean":"2020-11-16T22:30:06.042894+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:30:06.042894+0000","last_undegraded":"2020-11-16T22:30:06.042894+0000","last_fullsized":"2020-11-16T22:30:06.042894+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4465,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8663212,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4464,"num_write_kb":9023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"110'1036","reported_seq":"1180","reported_epoch":"117","state":"peering","last_fresh":"2020-11-16T22:30:07.046535+0000","last_change":"2020-11-16T22:30:07.044568+0000","last_active":"2020-11-16T22:30:07.044365+0000","last_peered":"2020-11-16T22:30:00.990045+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.019908+0000","last_became_peered":"2020-11-16T22:30:00.019908+0000","last_unstale":"2020-11-16T22:30:07.046535+0000","last_undegraded":"2020-11-16T22:30:07.046535+0000","last_fullsized":"2020-11-16T22:30:07.046535+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1036,"ondisk_log_size":1036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4160906,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":48,"num_read_kb":2465,"num_write":1231,"num_write_kb":5369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"46","reported_epoch":"117","state":"active+clean","last_fresh":"2020-11-16T22:30:07.047442+0000","last_change":"2020-11-16T22:30:07.047442+0000","last_active":"2020-11-16T22:30:07.047442+0000","last_peered":"2020-11-16T22:30:07.047442+0000","last_clean":"2020-11-16T22:30:07.047442+0000","last_became_active":"2020-11-16T22:30:07.047115+0000","last_became_peered":"2020-11-16T22:30:07.047115+0000","last_unstale":"2020-11-16T22:30:07.047442+0000","last_undegraded":"2020-11-16T22:30:07.047442+0000","last_fullsized":"2020-11-16T22:30:07.047442+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"44","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042734+0000","last_change":"2020-11-16T22:29:58.923416+0000","last_active":"2020-11-16T22:30:06.042734+0000","last_peered":"2020-11-16T22:30:06.042734+0000","last_clean":"2020-11-16T22:30:06.042734+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:30:06.042734+0000","last_undegraded":"2020-11-16T22:30:06.042734+0000","last_fullsized":"2020-11-16T22:30:06.042734+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"99'2177","reported_seq":"2229","reported_epoch":"112","state":"active+clean","last_fresh":"2020-11-16T22:30:02.027433+0000","last_change":"2020-11-16T22:30:02.027433+0000","last_active":"2020-11-16T22:30:02.027433+0000","last_peered":"2020-11-16T22:30:02.027433+0000","last_clean":"2020-11-16T22:30:02.027433+0000","last_became_active":"2020-11-16T22:30:02.026851+0000","last_became_peered":"2020-11-16T22:30:02.026851+0000","last_unstale":"2020-11-16T22:30:02.027433+0000","last_undegraded":"2020-11-16T22:30:02.027433+0000","last_fullsized":"2020-11-16T22:30:02.027433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2177,"ondisk_log_size":2177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2174,"num_write_kb":8467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1024","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.030106+0000","last_change":"2020-11-16T22:30:00.154019+0000","last_active":"2020-11-16T22:30:03.030106+0000","last_peered":"2020-11-16T22:30:03.030106+0000","last_clean":"2020-11-16T22:29:54.995625+0000","last_became_active":"2020-11-16T22:29:59.970950+0000","last_became_peered":"2020-11-16T22:29:59.970950+0000","last_unstale":"2020-11-16T22:30:03.030106+0000","last_undegraded":"2020-11-16T22:30:00.153785+0000","last_fullsized":"2020-11-16T22:29:59.911615+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":435},{"shards":"1","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042726+0000","last_change":"2020-11-16T22:29:58.923046+0000","last_active":"2020-11-16T22:30:06.042726+0000","last_peered":"2020-11-16T22:30:06.042726+0000","last_clean":"2020-11-16T22:30:06.042726+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:30:06.042726+0000","last_undegraded":"2020-11-16T22:30:06.042726+0000","last_fullsized":"2020-11-16T22:30:06.042726+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"191","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024949+0000","last_change":"2020-11-16T22:29:58.929640+0000","last_active":"2020-11-16T22:30:03.024949+0000","last_peered":"2020-11-16T22:30:03.024949+0000","last_clean":"2020-11-16T22:30:03.024949+0000","last_became_active":"2020-11-16T22:29:58.915283+0000","last_became_peered":"2020-11-16T22:29:58.915283+0000","last_unstale":"2020-11-16T22:30:03.024949+0000","last_undegraded":"2020-11-16T22:30:03.024949+0000","last_fullsized":"2020-11-16T22:30:03.024949+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":217756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"25772","reported_epoch":"113","state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2020-11-16T22:30:03.029245+0000","last_change":"2020-11-16T22:30:00.111256+0000","last_active":"2020-11-16T22:30:03.029245+0000","last_peered":"2020-11-16T22:30:03.029245+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.108653+0000","last_became_peered":"2020-11-16T22:30:00.108653+0000","last_unstale":"2020-11-16T22:30:03.029245+0000","last_undegraded":"2020-11-16T22:29:59.965211+0000","last_fullsized":"2020-11-16T22:29:59.912672+0000","mapping_epoch":109,"log_start":"93'1500","ondisk_log_start":"93'1500","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10054,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":463,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":463}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"11","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014898+0000","last_change":"2020-11-16T22:29:59.992177+0000","last_active":"2020-11-16T22:30:01.014898+0000","last_peered":"2020-11-16T22:30:01.014898+0000","last_clean":"2020-11-16T22:30:01.014898+0000","last_became_active":"2020-11-16T22:29:59.991752+0000","last_became_peered":"2020-11-16T22:29:59.991752+0000","last_unstale":"2020-11-16T22:30:01.014898+0000","last_undegraded":"2020-11-16T22:30:01.014898+0000","last_fullsized":"2020-11-16T22:30:01.014898+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"105'65","reported_seq":"23","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014608+0000","last_change":"2020-11-16T22:30:00.182675+0000","last_active":"2020-11-16T22:30:01.014608+0000","last_peered":"2020-11-16T22:30:01.014608+0000","last_clean":"2020-11-16T22:30:01.014608+0000","last_became_active":"2020-11-16T22:29:59.970355+0000","last_became_peered":"2020-11-16T22:29:59.970355+0000","last_unstale":"2020-11-16T22:30:01.014608+0000","last_undegraded":"2020-11-16T22:30:01.014608+0000","last_fullsized":"2020-11-16T22:30:01.014608+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":65,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"110'1131","reported_seq":"1308","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029271+0000","last_change":"2020-11-16T22:30:00.023326+0000","last_active":"2020-11-16T22:30:03.029271+0000","last_peered":"2020-11-16T22:30:03.029271+0000","last_clean":"2020-11-16T22:29:57.993024+0000","last_became_active":"2020-11-16T22:30:00.019682+0000","last_became_peered":"2020-11-16T22:30:00.019682+0000","last_unstale":"2020-11-16T22:30:03.029271+0000","last_undegraded":"2020-11-16T22:29:59.929587+0000","last_fullsized":"2020-11-16T22:29:59.912042+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1131,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2919944,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":128,"num_read_kb":4677,"num_write":1338,"num_write_kb":6129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":435},{"shards":"1","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"88'8","reported_seq":"23","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.015155+0000","last_change":"2020-11-16T22:30:00.122368+0000","last_active":"2020-11-16T22:30:01.015155+0000","last_peered":"2020-11-16T22:30:01.015155+0000","last_clean":"2020-11-16T22:30:01.015155+0000","last_became_active":"2020-11-16T22:29:59.957193+0000","last_became_peered":"2020-11-16T22:29:59.957193+0000","last_unstale":"2020-11-16T22:30:01.015155+0000","last_undegraded":"2020-11-16T22:30:01.015155+0000","last_fullsized":"2020-11-16T22:30:01.015155+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194879,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"11","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014754+0000","last_change":"2020-11-16T22:29:59.971632+0000","last_active":"2020-11-16T22:30:01.014754+0000","last_peered":"2020-11-16T22:30:01.014754+0000","last_clean":"2020-11-16T22:30:01.014754+0000","last_became_active":"2020-11-16T22:29:59.970132+0000","last_became_peered":"2020-11-16T22:29:59.970132+0000","last_unstale":"2020-11-16T22:30:01.014754+0000","last_undegraded":"2020-11-16T22:30:01.014754+0000","last_fullsized":"2020-11-16T22:30:01.014754+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"110'1170","reported_seq":"1456","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029345+0000","last_change":"2020-11-16T22:30:00.028176+0000","last_active":"2020-11-16T22:30:03.029345+0000","last_peered":"2020-11-16T22:30:03.029345+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.025087+0000","last_became_peered":"2020-11-16T22:30:00.025087+0000","last_unstale":"2020-11-16T22:30:03.029345+0000","last_undegraded":"2020-11-16T22:29:59.931596+0000","last_fullsized":"2020-11-16T22:29:59.912355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1170,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4801106,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":178,"num_read_kb":8919,"num_write":1343,"num_write_kb":5758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":456}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.044046+0000","last_change":"2020-11-16T22:29:58.922898+0000","last_active":"2020-11-16T22:30:06.044046+0000","last_peered":"2020-11-16T22:30:06.044046+0000","last_clean":"2020-11-16T22:30:06.044046+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:30:06.044046+0000","last_undegraded":"2020-11-16T22:30:06.044046+0000","last_fullsized":"2020-11-16T22:30:06.044046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"110'100","reported_seq":"36","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014777+0000","last_change":"2020-11-16T22:30:00.240943+0000","last_active":"2020-11-16T22:30:01.014777+0000","last_peered":"2020-11-16T22:30:01.014777+0000","last_clean":"2020-11-16T22:30:01.014777+0000","last_became_active":"2020-11-16T22:29:59.970416+0000","last_became_peered":"2020-11-16T22:29:59.970416+0000","last_unstale":"2020-11-16T22:30:01.014777+0000","last_undegraded":"2020-11-16T22:30:01.014777+0000","last_fullsized":"2020-11-16T22:30:01.014777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4194304,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":31766464,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":14464,"num_read_kb":364971,"num_write":23997,"num_write_kb":321550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":3696658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53698560,"data_stored":42071333,"data_compressed":3974568,"data_compressed_allocated":7569408,"data_compressed_original":15126222,"omap_allocated":0,"internal_metadata":0},"log_size":17237,"ondisk_log_size":17237,"up":16,"acting":10,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":55019724,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":12234,"num_write_kb":55699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":54892722,"num_keys_recovered":84,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65142784,"data_stored":105844501,"data_compressed":10257789,"data_compressed_allocated":40763392,"data_compressed_original":81526784,"omap_allocated":10534,"internal_metadata":0},"log_size":12237,"ondisk_log_size":12237,"up":16,"acting":14,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69012,"kb_used_data":36364,"kb_used_omap":26,"kb_used_meta":32613,"kb_avail":94302828,"statfs":{"total":96636764160,"available":96566095872,"internally_reserved":0,"allocated":37236736,"data_stored":61409505,"data_compressed":7370516,"data_compressed_allocated":28512256,"data_compressed_original":57024512,"omap_allocated":27325,"internal_metadata":33396035},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78024,"kb_used_data":12544,"kb_used_omap":0,"kb_used_meta":65472,"kb_avail":94293816,"statfs":{"total":96636764160,"available":96556867584,"internally_reserved":0,"allocated":12845056,"data_stored":9150856,"data_compressed":1029731,"data_compressed_allocated":2002944,"data_compressed_original":4005888,"omap_allocated":0,"internal_metadata":67043328},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:29:20 2020","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.65200000000000002,"15min":0.73199999999999998},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78000000000000003,"5min":1.028,"15min":1.028},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66400000000000003,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":195408,"kb_used_data":72328,"kb_used_omap":0,"kb_used_meta":123072,"kb_avail":94176432,"statfs":{"total":96636764160,"available":96436666368,"internally_reserved":0,"allocated":74063872,"data_stored":82968917,"data_compressed":6325008,"data_compressed_allocated":19914752,"data_compressed_original":39816910,"omap_allocated":0,"internal_metadata":126025728},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:29:23 2020","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":1.252,"15min":1.252},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]}]}} 2020-11-16T22:30:08.950 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:30:08.952 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:30:08.952 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:08.758+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:08.953 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:08.954 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:30:08.955 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:08.968+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:09.055 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:09.069+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:09.103 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:09.104 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:09.104 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:09.104 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:09.115 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:09.156 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:09.170+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:09.257 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:09.271+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:09.288 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:09.290 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:30:09.296 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:09.296 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:09.296 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:09.296 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:09.297 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:09.300 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2020-11-16T22:30:08.909626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86786188,"num_objects":3596,"num_object_clones":0,"num_object_copies":7192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":14490,"num_read_kb":365202,"num_write":36235,"num_write_kb":377269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":58589380,"num_keys_recovered":84,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29478,"ondisk_log_size":29478,"up":50,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":342444,"kb_used_data":121236,"kb_used_omap":26,"kb_used_meta":221157,"kb_avail":282773076,"statfs":{"total":289910292480,"available":289559629824,"internally_reserved":0,"allocated":124145664,"data_stored":153529278,"data_compressed":14725255,"data_compressed_allocated":50429952,"data_compressed_original":100847310,"omap_allocated":27325,"internal_metadata":226465091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-384,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":447120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.873595"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"1079","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029393+0000","last_change":"2020-11-16T22:30:00.040022+0000","last_active":"2020-11-16T22:30:03.029393+0000","last_peered":"2020-11-16T22:30:03.029393+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.035488+0000","last_became_peered":"2020-11-16T22:30:00.035488+0000","last_unstale":"2020-11-16T22:30:03.029393+0000","last_undegraded":"2020-11-16T22:29:59.951911+0000","last_fullsized":"2020-11-16T22:29:59.934996+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":455}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"109","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024965+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:30:03.024965+0000","last_peered":"2020-11-16T22:30:03.024965+0000","last_clean":"2020-11-16T22:30:03.024965+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:30:03.024965+0000","last_undegraded":"2020-11-16T22:30:03.024965+0000","last_fullsized":"2020-11-16T22:30:03.024965+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"97'1035","reported_seq":"1098","reported_epoch":"117","state":"remapped+peering","last_fresh":"2020-11-16T22:30:07.044309+0000","last_change":"2020-11-16T22:30:07.044517+0000","last_active":"2020-11-16T22:30:06.044220+0000","last_peered":"2020-11-16T22:30:05.037928+0000","last_clean":"2020-11-16T22:30:05.037928+0000","last_became_active":"2020-11-16T22:30:02.026496+0000","last_became_peered":"2020-11-16T22:30:02.026496+0000","last_unstale":"2020-11-16T22:30:07.044309+0000","last_undegraded":"2020-11-16T22:30:07.044309+0000","last_fullsized":"2020-11-16T22:30:07.044309+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1035,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388768,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1033,"num_write_kb":8391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8388858,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"2355","reported_epoch":"114","state":"active+undersized+remapped","last_fresh":"2020-11-16T22:30:04.313816+0000","last_change":"2020-11-16T22:30:04.314060+0000","last_active":"2020-11-16T22:30:04.313816+0000","last_peered":"2020-11-16T22:30:04.313816+0000","last_clean":"2020-11-16T22:29:54.995271+0000","last_became_active":"2020-11-16T22:29:59.981868+0000","last_became_peered":"2020-11-16T22:29:59.981868+0000","last_unstale":"2020-11-16T22:30:04.313816+0000","last_undegraded":"2020-11-16T22:30:04.313816+0000","last_fullsized":"2020-11-16T22:29:59.911749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":1876427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"1","objects":442}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.043156+0000","last_change":"2020-11-16T22:29:58.917676+0000","last_active":"2020-11-16T22:30:06.043156+0000","last_peered":"2020-11-16T22:30:06.043156+0000","last_clean":"2020-11-16T22:30:06.043156+0000","last_became_active":"2020-11-16T22:29:58.916780+0000","last_became_peered":"2020-11-16T22:29:58.916780+0000","last_unstale":"2020-11-16T22:30:06.043156+0000","last_undegraded":"2020-11-16T22:30:06.043156+0000","last_fullsized":"2020-11-16T22:30:06.043156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4421","reported_epoch":"117","state":"remapped+peering","last_fresh":"2020-11-16T22:30:07.043722+0000","last_change":"2020-11-16T22:30:07.043841+0000","last_active":"2020-11-16T22:30:06.043282+0000","last_peered":"2020-11-16T22:30:05.037854+0000","last_clean":"2020-11-16T22:30:05.037854+0000","last_became_active":"2020-11-16T22:30:02.024620+0000","last_became_peered":"2020-11-16T22:30:02.024620+0000","last_unstale":"2020-11-16T22:30:07.043722+0000","last_undegraded":"2020-11-16T22:30:07.043722+0000","last_fullsized":"2020-11-16T22:30:07.043722+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":16777250,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1111","reported_epoch":"116","state":"active+clean","last_fresh":"2020-11-16T22:30:06.043922+0000","last_change":"2020-11-16T22:30:04.039525+0000","last_active":"2020-11-16T22:30:06.043922+0000","last_peered":"2020-11-16T22:30:06.043922+0000","last_clean":"2020-11-16T22:30:06.043922+0000","last_became_active":"2020-11-16T22:30:04.035094+0000","last_became_peered":"2020-11-16T22:30:04.035094+0000","last_unstale":"2020-11-16T22:30:06.043922+0000","last_undegraded":"2020-11-16T22:30:06.043922+0000","last_fullsized":"2020-11-16T22:30:06.043922+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"109","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024953+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:30:03.024953+0000","last_peered":"2020-11-16T22:30:03.024953+0000","last_clean":"2020-11-16T22:30:03.024953+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:30:03.024953+0000","last_undegraded":"2020-11-16T22:30:03.024953+0000","last_fullsized":"2020-11-16T22:30:03.024953+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"110'4465","reported_seq":"4595","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042894+0000","last_change":"2020-11-16T22:30:02.027789+0000","last_active":"2020-11-16T22:30:06.042894+0000","last_peered":"2020-11-16T22:30:06.042894+0000","last_clean":"2020-11-16T22:30:06.042894+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:30:06.042894+0000","last_undegraded":"2020-11-16T22:30:06.042894+0000","last_fullsized":"2020-11-16T22:30:06.042894+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4465,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8663212,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4464,"num_write_kb":9023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"110'1036","reported_seq":"1180","reported_epoch":"117","state":"peering","last_fresh":"2020-11-16T22:30:07.046535+0000","last_change":"2020-11-16T22:30:07.044568+0000","last_active":"2020-11-16T22:30:07.044365+0000","last_peered":"2020-11-16T22:30:00.990045+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.019908+0000","last_became_peered":"2020-11-16T22:30:00.019908+0000","last_unstale":"2020-11-16T22:30:07.046535+0000","last_undegraded":"2020-11-16T22:30:07.046535+0000","last_fullsized":"2020-11-16T22:30:07.046535+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1036,"ondisk_log_size":1036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4160906,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":48,"num_read_kb":2465,"num_write":1231,"num_write_kb":5369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"46","reported_epoch":"117","state":"active+clean","last_fresh":"2020-11-16T22:30:07.047442+0000","last_change":"2020-11-16T22:30:07.047442+0000","last_active":"2020-11-16T22:30:07.047442+0000","last_peered":"2020-11-16T22:30:07.047442+0000","last_clean":"2020-11-16T22:30:07.047442+0000","last_became_active":"2020-11-16T22:30:07.047115+0000","last_became_peered":"2020-11-16T22:30:07.047115+0000","last_unstale":"2020-11-16T22:30:07.047442+0000","last_undegraded":"2020-11-16T22:30:07.047442+0000","last_fullsized":"2020-11-16T22:30:07.047442+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"44","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042734+0000","last_change":"2020-11-16T22:29:58.923416+0000","last_active":"2020-11-16T22:30:06.042734+0000","last_peered":"2020-11-16T22:30:06.042734+0000","last_clean":"2020-11-16T22:30:06.042734+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:30:06.042734+0000","last_undegraded":"2020-11-16T22:30:06.042734+0000","last_fullsized":"2020-11-16T22:30:06.042734+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"99'2177","reported_seq":"2229","reported_epoch":"112","state":"active+clean","last_fresh":"2020-11-16T22:30:02.027433+0000","last_change":"2020-11-16T22:30:02.027433+0000","last_active":"2020-11-16T22:30:02.027433+0000","last_peered":"2020-11-16T22:30:02.027433+0000","last_clean":"2020-11-16T22:30:02.027433+0000","last_became_active":"2020-11-16T22:30:02.026851+0000","last_became_peered":"2020-11-16T22:30:02.026851+0000","last_unstale":"2020-11-16T22:30:02.027433+0000","last_undegraded":"2020-11-16T22:30:02.027433+0000","last_fullsized":"2020-11-16T22:30:02.027433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2177,"ondisk_log_size":2177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2174,"num_write_kb":8467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1024","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.030106+0000","last_change":"2020-11-16T22:30:00.154019+0000","last_active":"2020-11-16T22:30:03.030106+0000","last_peered":"2020-11-16T22:30:03.030106+0000","last_clean":"2020-11-16T22:29:54.995625+0000","last_became_active":"2020-11-16T22:29:59.970950+0000","last_became_peered":"2020-11-16T22:29:59.970950+0000","last_unstale":"2020-11-16T22:30:03.030106+0000","last_undegraded":"2020-11-16T22:30:00.153785+0000","last_fullsized":"2020-11-16T22:29:59.911615+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":435},{"shards":"1","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.042726+0000","last_change":"2020-11-16T22:29:58.923046+0000","last_active":"2020-11-16T22:30:06.042726+0000","last_peered":"2020-11-16T22:30:06.042726+0000","last_clean":"2020-11-16T22:30:06.042726+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:30:06.042726+0000","last_undegraded":"2020-11-16T22:30:06.042726+0000","last_fullsized":"2020-11-16T22:30:06.042726+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"191","reported_epoch":"113","state":"active+clean","last_fresh":"2020-11-16T22:30:03.024949+0000","last_change":"2020-11-16T22:29:58.929640+0000","last_active":"2020-11-16T22:30:03.024949+0000","last_peered":"2020-11-16T22:30:03.024949+0000","last_clean":"2020-11-16T22:30:03.024949+0000","last_became_active":"2020-11-16T22:29:58.915283+0000","last_became_peered":"2020-11-16T22:29:58.915283+0000","last_unstale":"2020-11-16T22:30:03.024949+0000","last_undegraded":"2020-11-16T22:30:03.024949+0000","last_fullsized":"2020-11-16T22:30:03.024949+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":217756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"25772","reported_epoch":"113","state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2020-11-16T22:30:03.029245+0000","last_change":"2020-11-16T22:30:00.111256+0000","last_active":"2020-11-16T22:30:03.029245+0000","last_peered":"2020-11-16T22:30:03.029245+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.108653+0000","last_became_peered":"2020-11-16T22:30:00.108653+0000","last_unstale":"2020-11-16T22:30:03.029245+0000","last_undegraded":"2020-11-16T22:29:59.965211+0000","last_fullsized":"2020-11-16T22:29:59.912672+0000","mapping_epoch":109,"log_start":"93'1500","ondisk_log_start":"93'1500","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10054,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":463,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":463}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"11","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014898+0000","last_change":"2020-11-16T22:29:59.992177+0000","last_active":"2020-11-16T22:30:01.014898+0000","last_peered":"2020-11-16T22:30:01.014898+0000","last_clean":"2020-11-16T22:30:01.014898+0000","last_became_active":"2020-11-16T22:29:59.991752+0000","last_became_peered":"2020-11-16T22:29:59.991752+0000","last_unstale":"2020-11-16T22:30:01.014898+0000","last_undegraded":"2020-11-16T22:30:01.014898+0000","last_fullsized":"2020-11-16T22:30:01.014898+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"105'65","reported_seq":"23","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014608+0000","last_change":"2020-11-16T22:30:00.182675+0000","last_active":"2020-11-16T22:30:01.014608+0000","last_peered":"2020-11-16T22:30:01.014608+0000","last_clean":"2020-11-16T22:30:01.014608+0000","last_became_active":"2020-11-16T22:29:59.970355+0000","last_became_peered":"2020-11-16T22:29:59.970355+0000","last_unstale":"2020-11-16T22:30:01.014608+0000","last_undegraded":"2020-11-16T22:30:01.014608+0000","last_fullsized":"2020-11-16T22:30:01.014608+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":65,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"110'1131","reported_seq":"1308","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029271+0000","last_change":"2020-11-16T22:30:00.023326+0000","last_active":"2020-11-16T22:30:03.029271+0000","last_peered":"2020-11-16T22:30:03.029271+0000","last_clean":"2020-11-16T22:29:57.993024+0000","last_became_active":"2020-11-16T22:30:00.019682+0000","last_became_peered":"2020-11-16T22:30:00.019682+0000","last_unstale":"2020-11-16T22:30:03.029271+0000","last_undegraded":"2020-11-16T22:29:59.929587+0000","last_fullsized":"2020-11-16T22:29:59.912042+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1131,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2919944,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":128,"num_read_kb":4677,"num_write":1338,"num_write_kb":6129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":435},{"shards":"1","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"88'8","reported_seq":"23","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.015155+0000","last_change":"2020-11-16T22:30:00.122368+0000","last_active":"2020-11-16T22:30:01.015155+0000","last_peered":"2020-11-16T22:30:01.015155+0000","last_clean":"2020-11-16T22:30:01.015155+0000","last_became_active":"2020-11-16T22:29:59.957193+0000","last_became_peered":"2020-11-16T22:29:59.957193+0000","last_unstale":"2020-11-16T22:30:01.015155+0000","last_undegraded":"2020-11-16T22:30:01.015155+0000","last_fullsized":"2020-11-16T22:30:01.015155+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194879,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"11","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014754+0000","last_change":"2020-11-16T22:29:59.971632+0000","last_active":"2020-11-16T22:30:01.014754+0000","last_peered":"2020-11-16T22:30:01.014754+0000","last_clean":"2020-11-16T22:30:01.014754+0000","last_became_active":"2020-11-16T22:29:59.970132+0000","last_became_peered":"2020-11-16T22:29:59.970132+0000","last_unstale":"2020-11-16T22:30:01.014754+0000","last_undegraded":"2020-11-16T22:30:01.014754+0000","last_fullsized":"2020-11-16T22:30:01.014754+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"110'1170","reported_seq":"1456","reported_epoch":"113","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:30:03.029345+0000","last_change":"2020-11-16T22:30:00.028176+0000","last_active":"2020-11-16T22:30:03.029345+0000","last_peered":"2020-11-16T22:30:03.029345+0000","last_clean":"2020-11-16T22:23:17.119717+0000","last_became_active":"2020-11-16T22:30:00.025087+0000","last_became_peered":"2020-11-16T22:30:00.025087+0000","last_unstale":"2020-11-16T22:30:03.029345+0000","last_undegraded":"2020-11-16T22:29:59.931596+0000","last_fullsized":"2020-11-16T22:29:59.912355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1170,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4801106,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":178,"num_read_kb":8919,"num_write":1343,"num_write_kb":5758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":456}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"40","reported_epoch":"116","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:06.044046+0000","last_change":"2020-11-16T22:29:58.922898+0000","last_active":"2020-11-16T22:30:06.044046+0000","last_peered":"2020-11-16T22:30:06.044046+0000","last_clean":"2020-11-16T22:30:06.044046+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:30:06.044046+0000","last_undegraded":"2020-11-16T22:30:06.044046+0000","last_fullsized":"2020-11-16T22:30:06.044046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"110'100","reported_seq":"36","reported_epoch":"111","state":"active+clean","last_fresh":"2020-11-16T22:30:01.014777+0000","last_change":"2020-11-16T22:30:00.240943+0000","last_active":"2020-11-16T22:30:01.014777+0000","last_peered":"2020-11-16T22:30:01.014777+0000","last_clean":"2020-11-16T22:30:01.014777+0000","last_became_active":"2020-11-16T22:29:59.970416+0000","last_became_peered":"2020-11-16T22:29:59.970416+0000","last_unstale":"2020-11-16T22:30:01.014777+0000","last_undegraded":"2020-11-16T22:30:01.014777+0000","last_fullsized":"2020-11-16T22:30:01.014777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4194304,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":31766464,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":14464,"num_read_kb":364971,"num_write":23997,"num_write_kb":321550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":3696658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53698560,"data_stored":42071333,"data_compressed":3974568,"data_compressed_allocated":7569408,"data_compressed_original":15126222,"omap_allocated":0,"internal_metadata":0},"log_size":17237,"ondisk_log_size":17237,"up":16,"acting":10,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":55019724,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":12234,"num_write_kb":55699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":54892722,"num_keys_recovered":84,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65142784,"data_stored":105844501,"data_compressed":10257789,"data_compressed_allocated":40763392,"data_compressed_original":81526784,"omap_allocated":10534,"internal_metadata":0},"log_size":12237,"ondisk_log_size":12237,"up":16,"acting":14,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69012,"kb_used_data":36364,"kb_used_omap":26,"kb_used_meta":32613,"kb_avail":94302828,"statfs":{"total":96636764160,"available":96566095872,"internally_reserved":0,"allocated":37236736,"data_stored":61409505,"data_compressed":7370516,"data_compressed_allocated":28512256,"data_compressed_original":57024512,"omap_allocated":27325,"internal_metadata":33396035},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78024,"kb_used_data":12544,"kb_used_omap":0,"kb_used_meta":65472,"kb_avail":94293816,"statfs":{"total":96636764160,"available":96556867584,"internally_reserved":0,"allocated":12845056,"data_stored":9150856,"data_compressed":1029731,"data_compressed_allocated":2002944,"data_compressed_original":4005888,"omap_allocated":0,"internal_metadata":67043328},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:29:20 2020","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.65200000000000002,"15min":0.73199999999999998},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78000000000000003,"5min":1.028,"15min":1.028},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66400000000000003,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":195408,"kb_used_data":72328,"kb_used_omap":0,"kb_used_meta":123072,"kb_avail":94176432,"statfs":{"total":96636764160,"available":96436666368,"internally_reserved":0,"allocated":74063872,"data_stored":82968917,"data_compressed":6325008,"data_compressed_allocated":19914752,"data_compressed_original":39816910,"omap_allocated":0,"internal_metadata":126025728},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:29:23 2020","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":1.252,"15min":1.252},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]}]}} 2020-11-16T22:30:15.732 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:30:15.733 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:15.733 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:30:15.735 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:15.736 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:15.748+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:15.835 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:15.849+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:15.875 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 418.380041232, 2020-11-16T22:30:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:15.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 418.37992458899998, 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:15.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:16.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:16.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:16.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:16.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:16.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:16.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:16.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:16.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:16.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:16.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:16.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:16.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:16.506 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:16.506 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:16.506 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:16.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 418.37977622, 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:16.507 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:16.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:16.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:16.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:16.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:16.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:16.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:16.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:16.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:16.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:16.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:16.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 418.37899279099997, 2020-11-16T22:30:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:17.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:30:17.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:17.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:30:17.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:30:17.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:30:17.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:17.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:30:17.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:30:17.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:17.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:30:17.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 121.11068687300001, 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:17.342 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:17.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:17.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:17.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:17.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:17.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:17.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:17.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:17.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:17.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.755 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 119.63426017800001, 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:17.756 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:17.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:17.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:17.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:17.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:17.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:17.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:17.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:17.967 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 119.564037713, 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:17.968 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:17.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:17.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:17.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:18.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:18.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:18.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:18.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:18.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:18.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:18.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:18.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.91410198200001, 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:18.384 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:18.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:18.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:18.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:18.798 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:18.798 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.90835467399999, 2020-11-16T22:30:18.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:18.798 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:18.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:18.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:18.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:18.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:18.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:18.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:19.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:19.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:19.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:19.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:19.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:19.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:19.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.215 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.90721867000001, 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:19.216 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:19.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:19.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:19.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:19.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:19.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:19.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:19.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:19.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:19.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:19.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:19.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:19.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.630 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:19.630 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:19.630 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:19.630 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.630 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:19.631 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.632 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.90607157399999, 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:19.633 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:19.634 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:19.635 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:19.635 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.635 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:19.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:19.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:19.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:19.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:19.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:19.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:19.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:19.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:19.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:20.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:20.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:20.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.049 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.900130202, 2020-11-16T22:30:20.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:20.051 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.051 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:20.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:20.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:20.254 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:20.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.254 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:20.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:20.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:20.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:20.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:20.264 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.89778546300001, 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:20.265 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:20.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:20.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:20.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:20.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.89303932599999, 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.276 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:20.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:20.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:20.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.286 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.37417278300001, 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:20.287 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:20.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:20.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:30:20.295 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 117.855156017, 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:20.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:30:20.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 116.678348848, 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.304 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:20.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:20.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:20.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:20.312 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 116.661656646, 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:20.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:20.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 115.63247106999999, 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.321 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:20.529 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:20.530 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 7.8002563309999999, 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:20.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:20.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:20.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:20.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:20.537 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:20.538 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:20.540 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:20.540 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:20.540 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:20.541 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:20.541 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:20.541 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:20.541 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:30:20.541 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:20.542 INFO:teuthology.orchestra.run.smithi138.stdout:{"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 463/7192 objects degraded (6.438%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":125,"num_osds":3,"num_up_osds":3,"osd_up_since":1605565797,"num_in_osds":3,"osd_in_since":1605565389,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+clean+wait","count":3},{"state_name":"peering","count":1},{"state_name":"active+undersized+degraded+remapped+backfill_wait","count":1}],"num_pgs":25,"num_pools":4,"num_objects":3596,"data_bytes":87003437,"bytes_used":359305216,"bytes_avail":289550987264,"bytes_total":289910292480,"inactive_pgs_ratio":0.039999999105930328,"degraded_objects":463,"degraded_total":7192,"degraded_ratio":0.064377085650723032,"recovering_objects_per_sec":398,"recovering_bytes_per_sec":10031912,"recovering_keys_per_sec":19,"num_objects_recovered":1595,"num_bytes_recovered":40137402,"num_keys_recovered":79,"read_bytes_sec":98280,"write_bytes_sec":22010,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":4,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4212}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2020-11-16T22:27:51.688480+0000","services":{}},"progress_events":{}} 2020-11-16T22:30:20.542 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:20.553+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:20.542 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2020-11-16T22:30:20.543 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:30:20.543 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:20.640 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:20.654+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:20.680 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:20.680 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:20.689 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:20.741 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:20.755+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:20.814 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:20.814 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:20.823 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:20.841 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:30:20.843 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:20.857+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:20.853 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2020-11-16T22:30:19.720132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88003703,"num_objects":3596,"num_object_clones":0,"num_object_copies":7192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":14496,"num_read_kb":365894,"num_write":36250,"num_write_kb":377375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4715,"num_bytes_recovered":118547286,"num_keys_recovered":163,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29436,"ondisk_log_size":29436,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":364104,"kb_used_data":129200,"kb_used_omap":26,"kb_used_meta":234853,"kb_avail":282751416,"statfs":{"total":289910292480,"available":289537449984,"internally_reserved":0,"allocated":132300800,"data_stored":167108113,"data_compressed":16865468,"data_compressed_allocated":57638912,"data_compressed_original":115265230,"omap_allocated":27325,"internal_metadata":240489795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1000266,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-463,"num_objects_misplaced":-28,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":308,"num_write":10,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1354,"num_bytes_recovered":28029965,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001428"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"1084","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922705+0000","last_change":"2020-11-16T22:30:13.080333+0000","last_active":"2020-11-16T22:30:14.922705+0000","last_peered":"2020-11-16T22:30:14.922705+0000","last_clean":"2020-11-16T22:30:14.922705+0000","last_became_active":"2020-11-16T22:30:13.077423+0000","last_became_peered":"2020-11-16T22:30:13.077423+0000","last_unstale":"2020-11-16T22:30:14.922705+0000","last_undegraded":"2020-11-16T22:30:14.922705+0000","last_fullsized":"2020-11-16T22:30:14.922705+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"120","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922978+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:30:14.922978+0000","last_peered":"2020-11-16T22:30:14.922978+0000","last_clean":"2020-11-16T22:30:14.922978+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:30:14.922978+0000","last_undegraded":"2020-11-16T22:30:14.922978+0000","last_fullsized":"2020-11-16T22:30:14.922978+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"97'1035","reported_seq":"1133","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922898+0000","last_change":"2020-11-16T22:30:09.921128+0000","last_active":"2020-11-16T22:30:14.922898+0000","last_peered":"2020-11-16T22:30:14.922898+0000","last_clean":"2020-11-16T22:30:14.922898+0000","last_became_active":"2020-11-16T22:30:09.920593+0000","last_became_peered":"2020-11-16T22:30:09.920593+0000","last_unstale":"2020-11-16T22:30:14.922898+0000","last_undegraded":"2020-11-16T22:30:14.922898+0000","last_fullsized":"2020-11-16T22:30:14.922898+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1035,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388768,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1033,"num_write_kb":8391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"976","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923129+0000","last_change":"2020-11-16T22:30:07.052341+0000","last_active":"2020-11-16T22:30:14.923129+0000","last_peered":"2020-11-16T22:30:14.923129+0000","last_clean":"2020-11-16T22:30:14.923129+0000","last_became_active":"2020-11-16T22:30:07.049832+0000","last_became_peered":"2020-11-16T22:30:07.049832+0000","last_unstale":"2020-11-16T22:30:14.923129+0000","last_undegraded":"2020-11-16T22:30:14.923129+0000","last_fullsized":"2020-11-16T22:30:14.923129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":1876427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922700+0000","last_change":"2020-11-16T22:30:09.105464+0000","last_active":"2020-11-16T22:30:14.922700+0000","last_peered":"2020-11-16T22:30:14.922700+0000","last_clean":"2020-11-16T22:30:14.922700+0000","last_became_active":"2020-11-16T22:29:58.916780+0000","last_became_peered":"2020-11-16T22:29:58.916780+0000","last_unstale":"2020-11-16T22:30:14.922700+0000","last_undegraded":"2020-11-16T22:30:14.922700+0000","last_fullsized":"2020-11-16T22:30:14.922700+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4458","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923023+0000","last_change":"2020-11-16T22:30:09.922709+0000","last_active":"2020-11-16T22:30:14.923023+0000","last_peered":"2020-11-16T22:30:14.923023+0000","last_clean":"2020-11-16T22:30:14.923023+0000","last_became_active":"2020-11-16T22:30:09.921720+0000","last_became_peered":"2020-11-16T22:30:09.921720+0000","last_unstale":"2020-11-16T22:30:14.923023+0000","last_undegraded":"2020-11-16T22:30:14.923023+0000","last_fullsized":"2020-11-16T22:30:14.923023+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1119","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922903+0000","last_change":"2020-11-16T22:30:04.039525+0000","last_active":"2020-11-16T22:30:14.922903+0000","last_peered":"2020-11-16T22:30:14.922903+0000","last_clean":"2020-11-16T22:30:14.922903+0000","last_became_active":"2020-11-16T22:30:04.035094+0000","last_became_peered":"2020-11-16T22:30:04.035094+0000","last_unstale":"2020-11-16T22:30:14.922903+0000","last_undegraded":"2020-11-16T22:30:14.922903+0000","last_fullsized":"2020-11-16T22:30:14.922903+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"120","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922793+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:30:14.922793+0000","last_peered":"2020-11-16T22:30:14.922793+0000","last_clean":"2020-11-16T22:30:14.922793+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:30:14.922793+0000","last_undegraded":"2020-11-16T22:30:14.922793+0000","last_fullsized":"2020-11-16T22:30:14.922793+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"125'4472","reported_seq":"4611","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.126492+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:30:16.126492+0000","last_peered":"2020-11-16T22:30:16.126492+0000","last_clean":"2020-11-16T22:30:16.126492+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:30:16.126492+0000","last_undegraded":"2020-11-16T22:30:16.126492+0000","last_fullsized":"2020-11-16T22:30:16.126492+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4472,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8680052,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":9042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"125'1037","reported_seq":"1197","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.092038+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:30:16.092038+0000","last_peered":"2020-11-16T22:30:16.092038+0000","last_clean":"2020-11-16T22:30:16.092038+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:30:16.092038+0000","last_undegraded":"2020-11-16T22:30:16.092038+0000","last_fullsized":"2020-11-16T22:30:16.092038+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1037,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2924420,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":51,"num_read_kb":2849,"num_write":1232,"num_write_kb":5369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"54","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922865+0000","last_change":"2020-11-16T22:30:07.047442+0000","last_active":"2020-11-16T22:30:14.922865+0000","last_peered":"2020-11-16T22:30:14.922865+0000","last_clean":"2020-11-16T22:30:14.922865+0000","last_became_active":"2020-11-16T22:30:07.047115+0000","last_became_peered":"2020-11-16T22:30:07.047115+0000","last_unstale":"2020-11-16T22:30:14.922865+0000","last_undegraded":"2020-11-16T22:30:14.922865+0000","last_fullsized":"2020-11-16T22:30:14.922865+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"53","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922590+0000","last_change":"2020-11-16T22:30:09.100967+0000","last_active":"2020-11-16T22:30:14.922590+0000","last_peered":"2020-11-16T22:30:14.922590+0000","last_clean":"2020-11-16T22:30:14.922590+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:30:14.922590+0000","last_undegraded":"2020-11-16T22:30:14.922590+0000","last_fullsized":"2020-11-16T22:30:14.922590+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"99'2177","reported_seq":"2248","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922841+0000","last_change":"2020-11-16T22:30:09.922286+0000","last_active":"2020-11-16T22:30:14.922841+0000","last_peered":"2020-11-16T22:30:14.922841+0000","last_clean":"2020-11-16T22:30:14.922841+0000","last_became_active":"2020-11-16T22:30:09.921400+0000","last_became_peered":"2020-11-16T22:30:09.921400+0000","last_unstale":"2020-11-16T22:30:14.922841+0000","last_undegraded":"2020-11-16T22:30:14.922841+0000","last_fullsized":"2020-11-16T22:30:14.922841+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2177,"ondisk_log_size":2177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2174,"num_write_kb":8467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1007","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923208+0000","last_change":"2020-11-16T22:30:08.062471+0000","last_active":"2020-11-16T22:30:14.923208+0000","last_peered":"2020-11-16T22:30:14.923208+0000","last_clean":"2020-11-16T22:30:14.923208+0000","last_became_active":"2020-11-16T22:30:08.057493+0000","last_became_peered":"2020-11-16T22:30:08.057493+0000","last_unstale":"2020-11-16T22:30:14.923208+0000","last_undegraded":"2020-11-16T22:30:14.923208+0000","last_fullsized":"2020-11-16T22:30:14.923208+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922594+0000","last_change":"2020-11-16T22:30:09.102224+0000","last_active":"2020-11-16T22:30:14.922594+0000","last_peered":"2020-11-16T22:30:14.922594+0000","last_clean":"2020-11-16T22:30:14.922594+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:30:14.922594+0000","last_undegraded":"2020-11-16T22:30:14.922594+0000","last_fullsized":"2020-11-16T22:30:14.922594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"202","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922795+0000","last_change":"2020-11-16T22:29:58.929640+0000","last_active":"2020-11-16T22:30:14.922795+0000","last_peered":"2020-11-16T22:30:14.922795+0000","last_clean":"2020-11-16T22:30:14.922795+0000","last_became_active":"2020-11-16T22:29:58.915283+0000","last_became_peered":"2020-11-16T22:29:58.915283+0000","last_unstale":"2020-11-16T22:30:14.922795+0000","last_undegraded":"2020-11-16T22:30:14.922795+0000","last_fullsized":"2020-11-16T22:30:14.922795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":217756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"26720","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.929575+0000","last_change":"2020-11-16T22:30:14.928815+0000","last_active":"2020-11-16T22:30:14.929575+0000","last_peered":"2020-11-16T22:30:14.929575+0000","last_clean":"2020-11-16T22:30:14.929575+0000","last_became_active":"2020-11-16T22:30:14.926823+0000","last_became_peered":"2020-11-16T22:30:14.926823+0000","last_unstale":"2020-11-16T22:30:14.929575+0000","last_undegraded":"2020-11-16T22:30:14.929575+0000","last_fullsized":"2020-11-16T22:30:14.929575+0000","mapping_epoch":123,"log_start":"93'1554","ondisk_log_start":"93'1554","created":15,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10000,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"17","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923080+0000","last_change":"2020-11-16T22:30:07.050668+0000","last_active":"2020-11-16T22:30:14.923080+0000","last_peered":"2020-11-16T22:30:14.923080+0000","last_clean":"2020-11-16T22:30:14.923080+0000","last_became_active":"2020-11-16T22:30:07.050230+0000","last_became_peered":"2020-11-16T22:30:07.050230+0000","last_unstale":"2020-11-16T22:30:14.923080+0000","last_undegraded":"2020-11-16T22:30:14.923080+0000","last_fullsized":"2020-11-16T22:30:14.923080+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"125'66","reported_seq":"30","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.091515+0000","last_change":"2020-11-16T22:30:07.110028+0000","last_active":"2020-11-16T22:30:16.091515+0000","last_peered":"2020-11-16T22:30:16.091515+0000","last_clean":"2020-11-16T22:30:16.091515+0000","last_became_active":"2020-11-16T22:30:07.049365+0000","last_became_peered":"2020-11-16T22:30:07.049365+0000","last_unstale":"2020-11-16T22:30:16.091515+0000","last_undegraded":"2020-11-16T22:30:16.091515+0000","last_fullsized":"2020-11-16T22:30:16.091515+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":66,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"125'1132","reported_seq":"2656","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.124269+0000","last_change":"2020-11-16T22:30:10.120630+0000","last_active":"2020-11-16T22:30:16.124269+0000","last_peered":"2020-11-16T22:30:16.124269+0000","last_clean":"2020-11-16T22:30:16.124269+0000","last_became_active":"2020-11-16T22:30:08.211598+0000","last_became_peered":"2020-11-16T22:30:08.211598+0000","last_unstale":"2020-11-16T22:30:16.124269+0000","last_undegraded":"2020-11-16T22:30:16.124269+0000","last_fullsized":"2020-11-16T22:30:16.124269+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1132,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5139856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":131,"num_read_kb":4985,"num_write":1339,"num_write_kb":6129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":2884179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"88'8","reported_seq":"29","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923164+0000","last_change":"2020-11-16T22:30:07.172274+0000","last_active":"2020-11-16T22:30:14.923164+0000","last_peered":"2020-11-16T22:30:14.923164+0000","last_clean":"2020-11-16T22:30:14.923164+0000","last_became_active":"2020-11-16T22:30:07.052682+0000","last_became_peered":"2020-11-16T22:30:07.052682+0000","last_unstale":"2020-11-16T22:30:14.923164+0000","last_undegraded":"2020-11-16T22:30:14.923164+0000","last_fullsized":"2020-11-16T22:30:14.923164+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194879,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"17","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923112+0000","last_change":"2020-11-16T22:30:07.054250+0000","last_active":"2020-11-16T22:30:14.923112+0000","last_peered":"2020-11-16T22:30:14.923112+0000","last_clean":"2020-11-16T22:30:14.923112+0000","last_became_active":"2020-11-16T22:30:07.053851+0000","last_became_peered":"2020-11-16T22:30:07.053851+0000","last_unstale":"2020-11-16T22:30:14.923112+0000","last_undegraded":"2020-11-16T22:30:14.923112+0000","last_fullsized":"2020-11-16T22:30:14.923112+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"116'1171","reported_seq":"1655","reported_epoch":"124","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:14.922683+0000","last_change":"2020-11-16T22:30:08.922319+0000","last_active":"2020-11-16T22:30:14.922683+0000","last_peered":"2020-11-16T22:30:14.922683+0000","last_clean":"2020-11-16T22:30:14.922683+0000","last_became_active":"2020-11-16T22:30:08.918916+0000","last_became_peered":"2020-11-16T22:30:08.918916+0000","last_unstale":"2020-11-16T22:30:14.922683+0000","last_undegraded":"2020-11-16T22:30:14.922683+0000","last_fullsized":"2020-11-16T22:30:14.922683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1171,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5018355,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":178,"num_read_kb":8919,"num_write":1346,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923026+0000","last_change":"2020-11-16T22:30:09.100836+0000","last_active":"2020-11-16T22:30:14.923026+0000","last_peered":"2020-11-16T22:30:14.923026+0000","last_clean":"2020-11-16T22:30:14.923026+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:30:14.923026+0000","last_undegraded":"2020-11-16T22:30:14.923026+0000","last_fullsized":"2020-11-16T22:30:14.923026+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"114'101","reported_seq":"57","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923127+0000","last_change":"2020-11-16T22:30:09.920140+0000","last_active":"2020-11-16T22:30:14.923127+0000","last_peered":"2020-11-16T22:30:14.923127+0000","last_clean":"2020-11-16T22:30:14.923127+0000","last_became_active":"2020-11-16T22:30:09.919789+0000","last_became_peered":"2020-11-16T22:30:09.919789+0000","last_unstale":"2020-11-16T22:30:14.923127+0000","last_undegraded":"2020-11-16T22:30:14.923127+0000","last_fullsized":"2020-11-16T22:30:14.923127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":101,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8388608,"num_keys_recovered":143,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":32967139,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":14470,"num_read_kb":365663,"num_write":24002,"num_write_kb":321636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4663,"num_bytes_recovered":25905634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67035136,"data_stored":55453048,"data_compressed":4785585,"data_compressed_allocated":9216000,"data_compressed_original":18419406,"omap_allocated":0,"internal_metadata":0},"log_size":17186,"ondisk_log_size":17186,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":55036564,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":12244,"num_write_kb":55719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":92641652,"num_keys_recovered":163,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61718528,"data_stored":110073128,"data_compressed":12079883,"data_compressed_allocated":48422912,"data_compressed_original":96845824,"omap_allocated":10534,"internal_metadata":0},"log_size":12246,"ondisk_log_size":12246,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467973,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86276,"kb_used_data":47228,"kb_used_omap":26,"kb_used_meta":39013,"kb_avail":94285564,"statfs":{"total":96636764160,"available":96548417536,"internally_reserved":0,"allocated":48361472,"data_stored":62761204,"data_compressed":6659424,"data_compressed_allocated":22937600,"data_compressed_original":45875200,"omap_allocated":27325,"internal_metadata":39949635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103180,"kb_used_data":35204,"kb_used_omap":0,"kb_used_meta":67968,"kb_avail":94268660,"statfs":{"total":96636764160,"available":96531107840,"internally_reserved":0,"allocated":36048896,"data_stored":55337908,"data_compressed":6804485,"data_compressed_allocated":25071616,"data_compressed_original":50143232,"omap_allocated":0,"internal_metadata":69599232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:29:20 2020","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.65200000000000002,"15min":0.73199999999999998},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78000000000000003,"5min":1.028,"15min":1.028},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66400000000000003,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174648,"kb_used_data":46768,"kb_used_omap":0,"kb_used_meta":127872,"kb_avail":94197192,"statfs":{"total":96636764160,"available":96457924608,"internally_reserved":0,"allocated":47890432,"data_stored":49009001,"data_compressed":3401559,"data_compressed_allocated":9629696,"data_compressed_original":19246798,"omap_allocated":0,"internal_metadata":130940928},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:29:23 2020","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":1.252,"15min":1.252},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]}]}} 2020-11-16T22:30:20.945 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:20.958+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:20.947 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:20.947 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:20.956 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:21.045 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:21.059+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:21.098 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.099 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.108 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:21.152 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:21.166+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:21.234 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.234 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.243 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:21.247 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:21.261+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:21.348 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:21.362+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:21.405 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.406 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.414 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:21.458 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:21.472+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:21.549 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:21.563+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.552 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.562 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:21.650 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:21.665+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:21.726 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.726 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.735 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:21.751 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:21.766+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:21.853 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:21.867+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:21.860 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:21.861 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:21.869 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:21.953 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:21.967+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:22.006 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.006 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.015 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:22.054 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:22.068+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:22.141 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.141 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.150 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:22.155 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:22.169+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:22.255 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:22.270+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:22.274 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.274 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.283 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:22.356 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:22.371+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:22.418 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.418 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.427 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:22.457 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:22.471+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:22.542 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:22.543 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:30:22.550 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.550 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.558 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:22.572+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:22.559 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:22.659 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:22.673+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:22.700 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:22.720 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:22.723 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:22.723 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:22.732 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:22.740 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:22.741 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:22.741 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:22.741 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:22.761 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:22.775+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:22.862 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:22.876+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:22.862 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:22.862 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:22.862 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:22.863 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:22.871 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:22.915 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:30:22.962 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:22.977+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:22.996 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:22.997 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:22.997 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:22.997 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:22.997 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:23.005 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:23.063 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:23.078+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.132 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 425.63671630300001, 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:23.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:23.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:23.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.143 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 425.63659966, 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:23.144 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:23.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:23.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:23.155 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 425.63645129100001, 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:23.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:23.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:23.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:23.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 425.63566786199999, 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994, 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509372+0000", 2020-11-16T22:30:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010829599999999999 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509481+0000", 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1378000000000001e-05 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509502+0000", 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2099e-05 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509524+0000", 2020-11-16T22:30:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.509532+0000", 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036686710000000001 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513200+0000", 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011398799999999999 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513314+0000", 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031556299999999999 2020-11-16T22:30:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513630+0000", 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024077830000000001 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516038+0000", 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025562369999999998 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518594+0000", 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1297000000000001e-05 2020-11-16T22:30:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518635+0000", 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8673000000000001e-05 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518664+0000", 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0092917569999999994 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 128.36736194400001, 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:23.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.184 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.89093524899999, 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:23.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:23.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.82071278399999, 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.193 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:23.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:23.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:23.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:23.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:23.202 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.17077705299999, 2020-11-16T22:30:23.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:23.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:23.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:23.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:23.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:23.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:23.412 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.165029745, 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.413 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:23.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:23.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:23.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.163893741, 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:23.423 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:23.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:23.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:23.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:23.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.162746645, 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:23.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:23.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:23.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:23.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:23.444 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.156805273, 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:23.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:23.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:23.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:23.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.15446053399999, 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.455 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:23.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:23.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:23.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:23.465 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 126.149714397, 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:23.466 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:23.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:23.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 125.630847854, 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.476 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:23.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:23.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:23.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:791 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'7, mlcod=88'5)", 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 125.111831088, 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001, 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.484 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033209+0000", 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066954899999999997 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033879+0000", 2020-11-16T22:30:23.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1909999999999999e-06 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033884+0000", 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7790000000000001e-06 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033893+0000", 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033896+0000", 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1827000000000001e-05 2020-11-16T22:30:23.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033978+0000", 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5634e-05 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.033994+0000", 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085820949999999997 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042576+0000", 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4690999999999994e-05 2020-11-16T22:30:23.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:18.042641+0000", 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094316130000000001 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.491 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.694 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:23.694 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:23.694 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 123.935023919, 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.697 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:23.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.702 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 123.918331717, 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:23.704 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:23.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:23.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.710 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 122.889146141, 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:23.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:23.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:23.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 15.056931402, 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:23.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:23.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:23.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:23.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:23.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:23.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:23.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:23.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:23.726 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:23.728 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:23.742+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:23.728 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:23.828 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:23.843+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:23.855 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:30:23.865 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:23.865 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:23.866 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:30:23.866 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:23.875 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:23.930 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:23.944+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:24.031 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:24.045+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:24.042 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:30:24.043 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:30:24.043 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:24.053 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:24.132 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:24.147+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:24.172 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:30:24.183 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2020-11-16T22:30:23.721145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88003703,"num_objects":3596,"num_object_clones":0,"num_object_copies":7192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":14496,"num_read_kb":365894,"num_write":36250,"num_write_kb":377375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4715,"num_bytes_recovered":118547286,"num_keys_recovered":163,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29436,"ondisk_log_size":29436,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":364104,"kb_used_data":129200,"kb_used_omap":26,"kb_used_meta":234853,"kb_avail":282751416,"statfs":{"total":289910292480,"available":289537449984,"internally_reserved":0,"allocated":132300800,"data_stored":167108113,"data_compressed":16865468,"data_compressed_allocated":57638912,"data_compressed_original":115265230,"omap_allocated":27325,"internal_metadata":240489795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1000266,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-463,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":308,"num_write":10,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":14057774,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001936"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"1084","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922705+0000","last_change":"2020-11-16T22:30:13.080333+0000","last_active":"2020-11-16T22:30:14.922705+0000","last_peered":"2020-11-16T22:30:14.922705+0000","last_clean":"2020-11-16T22:30:14.922705+0000","last_became_active":"2020-11-16T22:30:13.077423+0000","last_became_peered":"2020-11-16T22:30:13.077423+0000","last_unstale":"2020-11-16T22:30:14.922705+0000","last_undegraded":"2020-11-16T22:30:14.922705+0000","last_fullsized":"2020-11-16T22:30:14.922705+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"120","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922978+0000","last_change":"2020-11-16T22:23:14.109848+0000","last_active":"2020-11-16T22:30:14.922978+0000","last_peered":"2020-11-16T22:30:14.922978+0000","last_clean":"2020-11-16T22:30:14.922978+0000","last_became_active":"2020-11-16T22:23:14.109527+0000","last_became_peered":"2020-11-16T22:23:14.109527+0000","last_unstale":"2020-11-16T22:30:14.922978+0000","last_undegraded":"2020-11-16T22:30:14.922978+0000","last_fullsized":"2020-11-16T22:30:14.922978+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"97'1035","reported_seq":"1133","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922898+0000","last_change":"2020-11-16T22:30:09.921128+0000","last_active":"2020-11-16T22:30:14.922898+0000","last_peered":"2020-11-16T22:30:14.922898+0000","last_clean":"2020-11-16T22:30:14.922898+0000","last_became_active":"2020-11-16T22:30:09.920593+0000","last_became_peered":"2020-11-16T22:30:09.920593+0000","last_unstale":"2020-11-16T22:30:14.922898+0000","last_undegraded":"2020-11-16T22:30:14.922898+0000","last_fullsized":"2020-11-16T22:30:14.922898+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":1035,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388768,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1033,"num_write_kb":8391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"976","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923129+0000","last_change":"2020-11-16T22:30:07.052341+0000","last_active":"2020-11-16T22:30:14.923129+0000","last_peered":"2020-11-16T22:30:14.923129+0000","last_clean":"2020-11-16T22:30:14.923129+0000","last_became_active":"2020-11-16T22:30:07.049832+0000","last_became_peered":"2020-11-16T22:30:07.049832+0000","last_unstale":"2020-11-16T22:30:14.923129+0000","last_undegraded":"2020-11-16T22:30:14.923129+0000","last_fullsized":"2020-11-16T22:30:14.923129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":1876427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922700+0000","last_change":"2020-11-16T22:30:09.105464+0000","last_active":"2020-11-16T22:30:14.922700+0000","last_peered":"2020-11-16T22:30:14.922700+0000","last_clean":"2020-11-16T22:30:14.922700+0000","last_became_active":"2020-11-16T22:29:58.916780+0000","last_became_peered":"2020-11-16T22:29:58.916780+0000","last_unstale":"2020-11-16T22:30:14.922700+0000","last_undegraded":"2020-11-16T22:30:14.922700+0000","last_fullsized":"2020-11-16T22:30:14.922700+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4458","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923023+0000","last_change":"2020-11-16T22:30:09.922709+0000","last_active":"2020-11-16T22:30:14.923023+0000","last_peered":"2020-11-16T22:30:14.923023+0000","last_clean":"2020-11-16T22:30:14.923023+0000","last_became_active":"2020-11-16T22:30:09.921720+0000","last_became_peered":"2020-11-16T22:30:09.921720+0000","last_unstale":"2020-11-16T22:30:14.923023+0000","last_undegraded":"2020-11-16T22:30:14.923023+0000","last_fullsized":"2020-11-16T22:30:14.923023+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1119","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922903+0000","last_change":"2020-11-16T22:30:04.039525+0000","last_active":"2020-11-16T22:30:14.922903+0000","last_peered":"2020-11-16T22:30:14.922903+0000","last_clean":"2020-11-16T22:30:14.922903+0000","last_became_active":"2020-11-16T22:30:04.035094+0000","last_became_peered":"2020-11-16T22:30:04.035094+0000","last_unstale":"2020-11-16T22:30:14.922903+0000","last_undegraded":"2020-11-16T22:30:14.922903+0000","last_fullsized":"2020-11-16T22:30:14.922903+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"120","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922793+0000","last_change":"2020-11-16T22:23:14.109810+0000","last_active":"2020-11-16T22:30:14.922793+0000","last_peered":"2020-11-16T22:30:14.922793+0000","last_clean":"2020-11-16T22:30:14.922793+0000","last_became_active":"2020-11-16T22:23:14.109422+0000","last_became_peered":"2020-11-16T22:23:14.109422+0000","last_unstale":"2020-11-16T22:30:14.922793+0000","last_undegraded":"2020-11-16T22:30:14.922793+0000","last_fullsized":"2020-11-16T22:30:14.922793+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"125'4472","reported_seq":"4611","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.126492+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:30:16.126492+0000","last_peered":"2020-11-16T22:30:16.126492+0000","last_clean":"2020-11-16T22:30:16.126492+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:30:16.126492+0000","last_undegraded":"2020-11-16T22:30:16.126492+0000","last_fullsized":"2020-11-16T22:30:16.126492+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4472,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8680052,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":9042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"125'1037","reported_seq":"1197","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.092038+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:30:16.092038+0000","last_peered":"2020-11-16T22:30:16.092038+0000","last_clean":"2020-11-16T22:30:16.092038+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:30:16.092038+0000","last_undegraded":"2020-11-16T22:30:16.092038+0000","last_fullsized":"2020-11-16T22:30:16.092038+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1037,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2924420,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":51,"num_read_kb":2849,"num_write":1232,"num_write_kb":5369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"54","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922865+0000","last_change":"2020-11-16T22:30:07.047442+0000","last_active":"2020-11-16T22:30:14.922865+0000","last_peered":"2020-11-16T22:30:14.922865+0000","last_clean":"2020-11-16T22:30:14.922865+0000","last_became_active":"2020-11-16T22:30:07.047115+0000","last_became_peered":"2020-11-16T22:30:07.047115+0000","last_unstale":"2020-11-16T22:30:14.922865+0000","last_undegraded":"2020-11-16T22:30:14.922865+0000","last_fullsized":"2020-11-16T22:30:14.922865+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"53","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922590+0000","last_change":"2020-11-16T22:30:09.100967+0000","last_active":"2020-11-16T22:30:14.922590+0000","last_peered":"2020-11-16T22:30:14.922590+0000","last_clean":"2020-11-16T22:30:14.922590+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:30:14.922590+0000","last_undegraded":"2020-11-16T22:30:14.922590+0000","last_fullsized":"2020-11-16T22:30:14.922590+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"99'2177","reported_seq":"2248","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922841+0000","last_change":"2020-11-16T22:30:09.922286+0000","last_active":"2020-11-16T22:30:14.922841+0000","last_peered":"2020-11-16T22:30:14.922841+0000","last_clean":"2020-11-16T22:30:14.922841+0000","last_became_active":"2020-11-16T22:30:09.921400+0000","last_became_peered":"2020-11-16T22:30:09.921400+0000","last_unstale":"2020-11-16T22:30:14.922841+0000","last_undegraded":"2020-11-16T22:30:14.922841+0000","last_fullsized":"2020-11-16T22:30:14.922841+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2177,"ondisk_log_size":2177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2174,"num_write_kb":8467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1007","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923208+0000","last_change":"2020-11-16T22:30:08.062471+0000","last_active":"2020-11-16T22:30:14.923208+0000","last_peered":"2020-11-16T22:30:14.923208+0000","last_clean":"2020-11-16T22:30:14.923208+0000","last_became_active":"2020-11-16T22:30:08.057493+0000","last_became_peered":"2020-11-16T22:30:08.057493+0000","last_unstale":"2020-11-16T22:30:14.923208+0000","last_undegraded":"2020-11-16T22:30:14.923208+0000","last_fullsized":"2020-11-16T22:30:14.923208+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922594+0000","last_change":"2020-11-16T22:30:09.102224+0000","last_active":"2020-11-16T22:30:14.922594+0000","last_peered":"2020-11-16T22:30:14.922594+0000","last_clean":"2020-11-16T22:30:14.922594+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:30:14.922594+0000","last_undegraded":"2020-11-16T22:30:14.922594+0000","last_fullsized":"2020-11-16T22:30:14.922594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"202","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.922795+0000","last_change":"2020-11-16T22:29:58.929640+0000","last_active":"2020-11-16T22:30:14.922795+0000","last_peered":"2020-11-16T22:30:14.922795+0000","last_clean":"2020-11-16T22:30:14.922795+0000","last_became_active":"2020-11-16T22:29:58.915283+0000","last_became_peered":"2020-11-16T22:29:58.915283+0000","last_unstale":"2020-11-16T22:30:14.922795+0000","last_undegraded":"2020-11-16T22:30:14.922795+0000","last_fullsized":"2020-11-16T22:30:14.922795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":217756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"26720","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.929575+0000","last_change":"2020-11-16T22:30:14.928815+0000","last_active":"2020-11-16T22:30:14.929575+0000","last_peered":"2020-11-16T22:30:14.929575+0000","last_clean":"2020-11-16T22:30:14.929575+0000","last_became_active":"2020-11-16T22:30:14.926823+0000","last_became_peered":"2020-11-16T22:30:14.926823+0000","last_unstale":"2020-11-16T22:30:14.929575+0000","last_undegraded":"2020-11-16T22:30:14.929575+0000","last_fullsized":"2020-11-16T22:30:14.929575+0000","mapping_epoch":123,"log_start":"93'1554","ondisk_log_start":"93'1554","created":15,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10000,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"17","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923080+0000","last_change":"2020-11-16T22:30:07.050668+0000","last_active":"2020-11-16T22:30:14.923080+0000","last_peered":"2020-11-16T22:30:14.923080+0000","last_clean":"2020-11-16T22:30:14.923080+0000","last_became_active":"2020-11-16T22:30:07.050230+0000","last_became_peered":"2020-11-16T22:30:07.050230+0000","last_unstale":"2020-11-16T22:30:14.923080+0000","last_undegraded":"2020-11-16T22:30:14.923080+0000","last_fullsized":"2020-11-16T22:30:14.923080+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"125'66","reported_seq":"30","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.091515+0000","last_change":"2020-11-16T22:30:07.110028+0000","last_active":"2020-11-16T22:30:16.091515+0000","last_peered":"2020-11-16T22:30:16.091515+0000","last_clean":"2020-11-16T22:30:16.091515+0000","last_became_active":"2020-11-16T22:30:07.049365+0000","last_became_peered":"2020-11-16T22:30:07.049365+0000","last_unstale":"2020-11-16T22:30:16.091515+0000","last_undegraded":"2020-11-16T22:30:16.091515+0000","last_fullsized":"2020-11-16T22:30:16.091515+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":66,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194947,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"125'1132","reported_seq":"2656","reported_epoch":"125","state":"active+clean","last_fresh":"2020-11-16T22:30:16.124269+0000","last_change":"2020-11-16T22:30:10.120630+0000","last_active":"2020-11-16T22:30:16.124269+0000","last_peered":"2020-11-16T22:30:16.124269+0000","last_clean":"2020-11-16T22:30:16.124269+0000","last_became_active":"2020-11-16T22:30:08.211598+0000","last_became_peered":"2020-11-16T22:30:08.211598+0000","last_unstale":"2020-11-16T22:30:16.124269+0000","last_undegraded":"2020-11-16T22:30:16.124269+0000","last_fullsized":"2020-11-16T22:30:16.124269+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1132,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5139856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":131,"num_read_kb":4985,"num_write":1339,"num_write_kb":6129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":2884179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"88'8","reported_seq":"29","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923164+0000","last_change":"2020-11-16T22:30:07.172274+0000","last_active":"2020-11-16T22:30:14.923164+0000","last_peered":"2020-11-16T22:30:14.923164+0000","last_clean":"2020-11-16T22:30:14.923164+0000","last_became_active":"2020-11-16T22:30:07.052682+0000","last_became_peered":"2020-11-16T22:30:07.052682+0000","last_unstale":"2020-11-16T22:30:14.923164+0000","last_undegraded":"2020-11-16T22:30:14.923164+0000","last_fullsized":"2020-11-16T22:30:14.923164+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4194879,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"17","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923112+0000","last_change":"2020-11-16T22:30:07.054250+0000","last_active":"2020-11-16T22:30:14.923112+0000","last_peered":"2020-11-16T22:30:14.923112+0000","last_clean":"2020-11-16T22:30:14.923112+0000","last_became_active":"2020-11-16T22:30:07.053851+0000","last_became_peered":"2020-11-16T22:30:07.053851+0000","last_unstale":"2020-11-16T22:30:14.923112+0000","last_undegraded":"2020-11-16T22:30:14.923112+0000","last_fullsized":"2020-11-16T22:30:14.923112+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"116'1171","reported_seq":"1655","reported_epoch":"124","state":"active+clean+wait","last_fresh":"2020-11-16T22:30:14.922683+0000","last_change":"2020-11-16T22:30:08.922319+0000","last_active":"2020-11-16T22:30:14.922683+0000","last_peered":"2020-11-16T22:30:14.922683+0000","last_clean":"2020-11-16T22:30:14.922683+0000","last_became_active":"2020-11-16T22:30:08.918916+0000","last_became_peered":"2020-11-16T22:30:08.918916+0000","last_unstale":"2020-11-16T22:30:14.922683+0000","last_undegraded":"2020-11-16T22:30:14.922683+0000","last_fullsized":"2020-11-16T22:30:14.922683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1171,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5018355,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":178,"num_read_kb":8919,"num_write":1346,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"49","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923026+0000","last_change":"2020-11-16T22:30:09.100836+0000","last_active":"2020-11-16T22:30:14.923026+0000","last_peered":"2020-11-16T22:30:14.923026+0000","last_clean":"2020-11-16T22:30:14.923026+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:30:14.923026+0000","last_undegraded":"2020-11-16T22:30:14.923026+0000","last_fullsized":"2020-11-16T22:30:14.923026+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"114'101","reported_seq":"57","reported_epoch":"124","state":"active+clean","last_fresh":"2020-11-16T22:30:14.923127+0000","last_change":"2020-11-16T22:30:09.920140+0000","last_active":"2020-11-16T22:30:14.923127+0000","last_peered":"2020-11-16T22:30:14.923127+0000","last_clean":"2020-11-16T22:30:14.923127+0000","last_became_active":"2020-11-16T22:30:09.919789+0000","last_became_peered":"2020-11-16T22:30:09.919789+0000","last_unstale":"2020-11-16T22:30:14.923127+0000","last_undegraded":"2020-11-16T22:30:14.923127+0000","last_fullsized":"2020-11-16T22:30:14.923127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":101,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8388608,"num_keys_recovered":143,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":32967139,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":14470,"num_read_kb":365663,"num_write":24002,"num_write_kb":321636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4663,"num_bytes_recovered":25905634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67035136,"data_stored":55453048,"data_compressed":4785585,"data_compressed_allocated":9216000,"data_compressed_original":18419406,"omap_allocated":0,"internal_metadata":0},"log_size":17186,"ondisk_log_size":17186,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":55036564,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":12244,"num_write_kb":55719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":92641652,"num_keys_recovered":163,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61718528,"data_stored":110073128,"data_compressed":12079883,"data_compressed_allocated":48422912,"data_compressed_original":96845824,"omap_allocated":10534,"internal_metadata":0},"log_size":12246,"ondisk_log_size":12246,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467974,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86276,"kb_used_data":47228,"kb_used_omap":26,"kb_used_meta":39013,"kb_avail":94285564,"statfs":{"total":96636764160,"available":96548417536,"internally_reserved":0,"allocated":48361472,"data_stored":62761204,"data_compressed":6659424,"data_compressed_allocated":22937600,"data_compressed_original":45875200,"omap_allocated":27325,"internal_metadata":39949635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103180,"kb_used_data":35204,"kb_used_omap":0,"kb_used_meta":67968,"kb_avail":94268660,"statfs":{"total":96636764160,"available":96531107840,"internally_reserved":0,"allocated":36048896,"data_stored":55337908,"data_compressed":6804485,"data_compressed_allocated":25071616,"data_compressed_original":50143232,"omap_allocated":0,"internal_metadata":69599232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:29:20 2020","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.65200000000000002,"15min":0.73199999999999998},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.78000000000000003,"5min":1.028,"15min":1.028},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66400000000000003,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174648,"kb_used_data":46768,"kb_used_omap":0,"kb_used_meta":127872,"kb_avail":94197192,"statfs":{"total":96636764160,"available":96457924608,"internally_reserved":0,"allocated":47890432,"data_stored":49009001,"data_compressed":3401559,"data_compressed_allocated":9629696,"data_compressed_original":19246798,"omap_allocated":0,"internal_metadata":130940928},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:29:23 2020","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.874,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":1.252,"15min":1.252},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]}]}} 2020-11-16T22:30:24.185 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-11-16T22:30:24.185 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:30:24.186 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:30:24.186 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2020-11-16T22:30:24.198 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.89036519199999, 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:30:24.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:30:24.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.209 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 133.88986230399999, 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:24.210 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:30:24.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:30:24.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:30:24.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.219 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 127.96119107200001, 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:24.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.224 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:30:24.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:30:24.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.229 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 127.243904601, 2020-11-16T22:30:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:24.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:30:24.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:30:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:30:24.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.999742388, 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.241 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:30:24.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:30:24.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:30:24.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:30:24.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.251 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998915604, 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:24.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:30:24.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:30:24.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:30:24.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:30:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:30:24.462 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:24.463 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998833316, 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:30:24.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:30:24.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:30:24.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998750972, 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.474 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:24.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:24.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:24.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:24.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:24.484 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.484 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.488 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.99867585200001, 2020-11-16T22:30:24.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:24.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:24.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:24.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998596918, 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.500 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:24.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:24.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:24.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:24.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:24.510 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998519496, 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:24.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:24.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:24.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:24.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.520 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998439334, 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:24.521 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998357521, 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998264135, 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.998143739, 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.99804367199999, 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 83.597770687999997, 2020-11-16T22:30:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:24.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 47.751797754999998, 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:24.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:24.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:24.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:24.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:24.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:24.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:24.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:25.000 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 25.300995614000001, 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:30:25.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:30:25.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:25.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:30:25.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:25.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 25.002618555000002, 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:25.010 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:25.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:30:25.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:25.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:25.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:25.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:30:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:30:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:30:25.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:30:25.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:25.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:25.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.224 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:25.225 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:25.225 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:25.225 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:25.226 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:30:25.228 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:30:25.228 INFO:teuthology.orchestra.run.smithi138.stdout:avg 16.6667 2020-11-16T22:30:25.228 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 0.471404 -> 0.471404 (expected baseline 3.33333) 2020-11-16T22:30:25.228 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.96 -> 0.96 * mean) 2020-11-16T22:30:25.228 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.0 with 17 -> 17 pgs (1.02 -> 1.02 * mean) 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0013 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0129 2020-11-16T22:30:25.229 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:30:25.230 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:30:25.434 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2020-11-16T22:30:25.435 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:25.435 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:25.240+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:25.437 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:25.450+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:25.537 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:25.551+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:25.575 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17703 3.0 3:1e8917d7:::200.0000000e:head [write 666073~1931 [fadvise_dontneed] in=1931b] snapc 0=[] ondisk+write+known_if_redirected+full_force e126)", 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:25.589273+0000", 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0013107290000000001, 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001329071, 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:30:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589273+0000", 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589273+0000", 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6300999999999997e-05 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589310+0000", 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7679999999999997e-06 2020-11-16T22:30:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589316+0000", 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1426999999999999e-05 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589327+0000", 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3709999999999999e-06 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589332+0000", 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8980000000000002e-05 2020-11-16T22:30:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589381+0000", 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3359000000000001e-05 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589445+0000", 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4036999999999994e-05 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:25.589539+0000", 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026524300000000002 2020-11-16T22:30:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:30:25.583 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:25.586 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:25.791 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:25.791 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:25.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:25.792 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:25.792 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:25.792 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:25.792 INFO:tasks.ceph.mgr.x.smithi138.stderr:2020-11-16T22:30:25.795+0000 7f410bec3700 -1 mgr.server reply reply (1) Operation not permitted no change 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:avg 16.6667 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 0.471404 -> 0.471404 (expected baseline 3.33333) 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.96 -> 0.96 * mean) 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.0 with 17 -> 17 pgs (1.02 -> 1.02 * mean) 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:25.793 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0013 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0014 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 1.0000 -> 0.9500 2020-11-16T22:30:25.794 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:30:25.795 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:25.805+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:25.795 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:30:25.796 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2020-11-16T22:30:25.891 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:25.905+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:25.929 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:25.929 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:25.938 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:30:25.939 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:25.992 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:26.006+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:26.081 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.081 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.091 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:26.093 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.108+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.146 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:avg 16.6667 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 0.471404 -> 0.471404 (expected baseline 3.33333) 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.96 -> 0.96 * mean) 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.0 with 17 -> 17 pgs (1.02 -> 1.02 * mean) 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:30:26.147 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:30:26.148 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:30:26.148 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0013 2020-11-16T22:30:26.148 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0129 2020-11-16T22:30:26.149 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:30:26.160 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:30:26.160 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2020-11-16T22:30:26.194 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.208+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.220 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.221 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.230 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:26.295 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:26.309+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:26.357 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.357 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.366 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:26.396 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.410+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.469 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:avg 16.6667 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 0.471404 -> 0.471404 (expected baseline 3.33333) 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.96 -> 0.96 * mean) 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.0 with 17 -> 17 pgs (1.02 -> 1.02 * mean) 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:30:26.470 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:30:26.471 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:30:26.471 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0013 2020-11-16T22:30:26.471 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0129 2020-11-16T22:30:26.472 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:30:26.482 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:30:26.483 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2020-11-16T22:30:26.496 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.511+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.538 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.539 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.548 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:26.597 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.612+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.680 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.680 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.690 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:26.731 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:26.746+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:26.792 INFO:tasks.ceph.mgr.x.smithi138.stderr:2020-11-16T22:30:26.806+0000 7f410bec3700 -1 mgr.server reply reply (1) Operation not permitted no change 2020-11-16T22:30:26.792 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:avg 16.6667 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 0.471404 -> 0.471404 (expected baseline 3.33333) 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.96 -> 0.96 * mean) 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.0 with 17 -> 17 pgs (1.02 -> 1.02 * mean) 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:30:26.793 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:30:26.794 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:30:26.794 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0013 2020-11-16T22:30:26.794 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0014 2020-11-16T22:30:26.794 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 1.0000 -> 0.9500 2020-11-16T22:30:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:30:26.799 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:26.814+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:26.863 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:26.864 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:26.874 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:26.901 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:26.915+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:27.002 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:27.016+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:27.014 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.014 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.024 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:27.102 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:27.117+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:27.161 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.161 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.172 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:27.203 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:27.218+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:27.305 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:27.319+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:27.347 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.347 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.357 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:27.406 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:27.420+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:27.506 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.507 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.507 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:27.521+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:27.518 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:27.607 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:27.622+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:27.648 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.648 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.658 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:27.709 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:27.723+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:27.787 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.788 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.793 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:27.797 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:27.842 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:27.856+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:27.909 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:27.923+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:27.931 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:27.931 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:27.941 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:27.942 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:27.950 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:27.983 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:27.984 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:28.010 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:28.025+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:28.082 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:28.082 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:28.083 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:30:28.083 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:28.092 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:28.111 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:28.126+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:28.212 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:28.226+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:28.291 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:28.291 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:28.291 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:30:28.292 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:30:28.292 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:28.313 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:28.327+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:28.414 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:28.428+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:28.489 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:28.489 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:30:28.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:30:28.489 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:28.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 143.18142114700001, 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:30:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:30:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:30:28.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:28.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:30:28.498 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:28.499 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.18091825900001, 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:30:28.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:30:28.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:30:28.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:30:28.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.508 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:28.509 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:28.509 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 132.25224702700001, 2020-11-16T22:30:28.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:30:28.509 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:28.710 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:28.710 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:28.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:28.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:30:28.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:30:28.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:28.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:30:28.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:30:28.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:28.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:30:28.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:30:28.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:28.922 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 131.53496055599999, 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:28.923 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:30:28.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:30:28.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:30:28.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:30:29.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:30:29.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:30:29.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:30:29.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.135 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.29079834300001, 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:30:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28997155900001, 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:30:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:30:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:30:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:30:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:30:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:30:29.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:30:29.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28988927099999, 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:30:29.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:30:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:30:29.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.289806927, 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:29.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:29.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:29.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:29.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:29.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:29.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28973180700001, 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:29.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:29.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28965287299999, 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:29.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:29.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:29.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:29.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:29.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:29.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:29.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:30.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28957545099999, 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:30.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:30.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:30.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:30.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:30.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:30.011 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.012 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:30.012 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:30.214 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.289495289, 2020-11-16T22:30:30.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:30.214 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:30.214 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:30.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:30.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:30.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:30.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:30.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:30.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:30.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:30.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:30.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:30.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:30.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:30.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.428 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:30.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:30.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.630 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28941347599999, 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:30.631 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:30.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:30.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:30.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:30.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:30.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:30.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:31.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:31.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:31.050 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.051 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:31.051 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.052 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:31.052 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:31.052 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28932008999999, 2020-11-16T22:30:31.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:31.052 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:31.053 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:31.053 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:31.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:31.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:31.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:31.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:31.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:31.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:31.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:31.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:31.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:31.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:31.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:31.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:31.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:31.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:31.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:31.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:31.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:31.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:31.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:31.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:31.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:31.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:31.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:31.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28919969399999, 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:31.468 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:31.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:31.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:31.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:31.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:31.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:31.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:31.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:31.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:31.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:31.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:31.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:31.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:31.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:31.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 129.28909962700001, 2020-11-16T22:30:31.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:31.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:31.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:31.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:31.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:31.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:32.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:32.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:32.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:32.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:32.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:32.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:32.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:32.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:32.300 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 87.888826643000002, 2020-11-16T22:30:32.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:32.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:32.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:32.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:32.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:32.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:32.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:32.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:32.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:32.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:32.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:32.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:32.717 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.042853710000003, 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:32.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:32.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:32.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:32.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:32.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:32.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:32.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:32.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:32.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:32.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:32.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:32.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:32.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:32.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:32.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:32.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:32.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:32.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:32.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:32.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:32.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:32.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:32.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:32.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:32.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:32.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:33.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:33.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:33.136 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.592051568999999, 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:30:33.137 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:33.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:30:33.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:30:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:30:33.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:30:33.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:33.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:30:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:30:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.550 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:33.550 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:30:33.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:30:33.550 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:33.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:30:33.552 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:30:33.552 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.552 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.293674509999999, 2020-11-16T22:30:33.553 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:33.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:30:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:30:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:33.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:30:33.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:30:33.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:30:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:33.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:30:33.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:30:33.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:30:33.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:30:33.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:33.970 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:33.971 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:33.972 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:30:33.972 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:30:34.174 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.430760 2020-11-16T22:30:34.174 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.43076027395468053 2020-11-16T22:30:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:30:34.177 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:34.178 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:33.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:34.217 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:34.232+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:34.280 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:34.294+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:34.321 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:34.321 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:34.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.321 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:31569 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1953792~16384 [358@4194304] in=16384b] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:34.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:34.332873+0000", 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0030013850000000001, 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030199110000000001, 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:34.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 31569 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:34.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332873+0000", 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332873+0000", 2020-11-16T22:30:34.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2703999999999999e-05 2020-11-16T22:30:34.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:34.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332966+0000", 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6840000000000002e-06 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332975+0000", 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5925000000000001e-05 2020-11-16T22:30:34.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332991+0000", 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1450000000000002e-06 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.332998+0000", 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8959000000000003e-05 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.333057+0000", 2020-11-16T22:30:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4988000000000006e-05 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.333152+0000", 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015553999999999999 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:34.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.333307+0000", 2020-11-16T22:30:34.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001616543 2020-11-16T22:30:34.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:34.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:34.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:34.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.334924+0000", 2020-11-16T22:30:34.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020504880000000001 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:30:34.735 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:34.738 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:34.738 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:34.738 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:34.738 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:34.738 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:34.751+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:34.739 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:34.838 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:34.852+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:34.914 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:34.914 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:34.925 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:34.938 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:34.953+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:34.960 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.1 primary-affinity to 0.43076 (8282302) 2020-11-16T22:30:35.039 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:35.053+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:35.051 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.051 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.060 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:35.263 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:35.278+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:35.364 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:35.378+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:35.389 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.389 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.398 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:35.464 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:35.479+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:35.522 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.522 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.531 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:35.565 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:35.580+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:35.658 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.659 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.666 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:35.680+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:35.668 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:35.767 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:35.781+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:35.798 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.798 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.807 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:35.867 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:35.882+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:35.932 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:35.932 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:35.941 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:35.968 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:35.982+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:36.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:36.083+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:36.121 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.121 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.132 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:36.169 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:36.184+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:36.177 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:30:36.270 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:36.285+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:36.275 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.276 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.286 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:36.371 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:36.386+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:36.423 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.423 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.434 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:36.473 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:36.487+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:36.573 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.574 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.574 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:36.588+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:36.584 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:36.675 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:36.689+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:36.716 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.716 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.726 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:36.739 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:36.776 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:36.790+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:36.854 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:36.854 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:36.863 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:36.877 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:36.891+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:36.885 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:36.906 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:36.927 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:36.928 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:36.928 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:36.928 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:36.972 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:30:36.977 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:36.991+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:37.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 439.54101326300002, 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:37.037 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:37.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:37.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:37.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:37.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:37.048 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:37.049 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 439.54089662000001, 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:37.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:37.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:37.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:37.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:37.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 439.54074825100002, 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.062 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:37.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:37.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:37.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:37.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.27165890399999, 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.075 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:37.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:37.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:37.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:37.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.79523220900001, 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.084 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:37.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:37.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.092 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.725009744, 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.093 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:37.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:37.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:37.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:37.104 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.07507401300001, 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:37.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:37.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:37.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.113 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.06932670500001, 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:37.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:37.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:37.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:37.124 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:37.125 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.06819070099999, 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.126 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:37.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:37.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:37.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:37.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:37.137 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.06704360500001, 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.138 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:37.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:37.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:37.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.148 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.06110223300001, 2020-11-16T22:30:37.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.150 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:37.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:37.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:37.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:37.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:37.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.160 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:37.161 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.05875749399999, 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.162 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:37.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:37.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.172 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 140.05401135700001, 2020-11-16T22:30:37.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.174 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:37.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:37.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:37.185 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.53514481400001, 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:37.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:37.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 137.83932087900001, 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:37.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:37.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:37.202 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:37.203 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 137.82262867700001, 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:37.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:37.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:37.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 136.79344310100001, 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.212 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:37.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:37.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:37.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 28.961228362, 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.221 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:37.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18638 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 259@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 6.4072592259999999, 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997, 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0172e-05 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642128+0000", 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1904e-05 2020-11-16T22:30:37.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642140+0000", 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642155+0000", 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0720000000000002e-06 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642162+0000", 2020-11-16T22:30:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3480999999999997e-05 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642235+0000", 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7229999999999999e-05 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642293+0000", 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3079999999999997e-05 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642376+0000", 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012180730000000001 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.643594+0000", 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040679215999999997 2020-11-16T22:30:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684273+0000", 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9913000000000001e-05 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684323+0000", 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5922999999999999e-05 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684349+0000", 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997 2020-11-16T22:30:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19235 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 2.840778786, 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619, 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3933999999999997e-05 2020-11-16T22:30:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208592+0000", 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208600+0000", 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0356e-05 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208610+0000", 2020-11-16T22:30:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3980000000000001e-06 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208614+0000", 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4449000000000001e-05 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208659+0000", 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6623e-05 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208756+0000", 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6582000000000006e-05 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208842+0000", 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091079300000000002 2020-11-16T22:30:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.209753+0000", 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096023089999999998 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219355+0000", 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.566e-05 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219401+0000", 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6072000000000001e-05 2020-11-16T22:30:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219457+0000", 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:37.250 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:37.251 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:37.252 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:37.265+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:37.352 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:37.366+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:37.403 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:37.403 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:19621 3.0 e131/111 3:1e8917d7:::200.0000000e:head v 131'5973, mlcod=131'5971)", 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:37.418283+0000", 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00027204699999999998, 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00028821800000000002, 2020-11-16T22:30:37.404 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418283+0000", 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418283+0000", 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4671000000000001e-05 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418318+0000", 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6989999999999997e-06 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418324+0000", 2020-11-16T22:30:37.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2019999999999993e-06 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418334+0000", 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3839999999999996e-06 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418339+0000", 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9841e-05 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418379+0000", 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.951e-05 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:37.418408+0000", 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012530700000000001 2020-11-16T22:30:37.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:30:37.411 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:37.413 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:37.457 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:37.471+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:37.548 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:30:37.548 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:30:37.548 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:37.553 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:37.568+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:37.558 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:37.654 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:37.669+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:37.748 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:37.748 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:37.749 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 152.44109727700001, 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:37.750 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:37.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:30:37.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:30:37.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:30:37.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:37.759 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 147.44059438900001, 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:37.760 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:30:37.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:30:37.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:37.769 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 141.51192315700001, 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:37.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:30:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:30:37.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:30:37.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 140.79463668599999, 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:30:37.780 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:30:37.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:30:37.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:30:37.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:30:37.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.790 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.55047447300001, 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:30:37.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:30:37.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:30:37.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54964768900001, 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.801 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:30:37.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:30:37.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:30:37.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:30:37.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.811 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.549565401, 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:30:37.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:30:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:30:37.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.549483057, 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:30:37.822 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:37.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:37.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:37.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.832 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54940793700001, 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:37.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:37.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:37.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:37.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.843 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.843 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.843 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.844 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:37.844 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.549329003, 2020-11-16T22:30:37.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:37.844 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.844 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.845 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:37.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:37.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:37.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:37.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:37.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54925158099999, 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.855 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:37.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:37.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.865 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.549171419, 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.866 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:37.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:37.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:37.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:37.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.875 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.549089606, 2020-11-16T22:30:37.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:37.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:37.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54899621999999, 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.887 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:37.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:37.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:37.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54887582399999, 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.898 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:37.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:37.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.908 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 138.54877575699999, 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:37.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:37.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:37.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:37.919 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 97.148502773000004, 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:37.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:37.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:37.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:37.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.929 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.302529839999998, 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:37.930 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:37.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:37.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:37.938 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:37.939 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 38.851727699000001, 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:30:37.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:30:37.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:30:37.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:30:37.948 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:37.949 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 38.553350639999998, 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:37.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:30:37.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:37.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:30:37.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:37.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:37.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:30:37.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:30:37.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:37.961 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:37.962 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:37.963 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:37.976+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:38.063 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:38.077+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:38.100 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19739 3.0 3:1e8917d7:::200.0000000e:head [write 3353620~1931 [fadvise_dontneed] in=1931b] snapc 0=[] ondisk+write+known_if_redirected+full_force e131)", 2020-11-16T22:30:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:38.113087+0000", 2020-11-16T22:30:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0026930280000000001, 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002711373, 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113087+0000", 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113087+0000", 2020-11-16T22:30:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7219999999999999e-05 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113124+0000", 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1290000000000001e-06 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113130+0000", 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0154e-05 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113140+0000", 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7459999999999998e-06 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113144+0000", 2020-11-16T22:30:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6501000000000002e-05 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113180+0000", 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5061999999999995e-05 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113245+0000", 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5922999999999996e-05 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.113341+0000", 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019020689999999999 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:38.115243+0000", 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002156804 2020-11-16T22:30:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:30:38.110 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:38.111 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:38.164 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:38.178+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:38.243 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:30:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:30:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:30:38.244 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:38.253 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:38.265 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:38.278+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:38.365 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:38.380+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:38.431 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:38.431 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:30:38.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.444421161000001, 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:30:38.432 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:38.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:30:38.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:30:38.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:30:38.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.441 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.236270994000002, 2020-11-16T22:30:38.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:38.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:30:38.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:30:38.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:30:38.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:30:38.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:30:38.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:30:38.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.456 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.415920563, 2020-11-16T22:30:38.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001, 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025738089999999998 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031142+0000", 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.242e-05 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031155+0000", 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6178e-05 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031171+0000", 2020-11-16T22:30:38.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1250000000000004e-06 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031178+0000", 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5498000000000002e-05 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031243+0000", 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1666e-05 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031255+0000", 2020-11-16T22:30:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099647690000000001 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.041220+0000", 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.464 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.394465492999998, 2020-11-16T22:30:38.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293, 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:38.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010396590000000001 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051063+0000", 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1077000000000002e-05 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051084+0000", 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6332000000000001e-05 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051111+0000", 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2245e-05 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051123+0000", 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000182122 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051305+0000", 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2546e-05 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051318+0000", 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011890312 2020-11-16T22:30:38.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.063208+0000", 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.376837533, 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927, 2020-11-16T22:30:38.473 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001385074 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069037+0000", 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9641999999999999e-05 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069056+0000", 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0559000000000001e-05 2020-11-16T22:30:38.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069077+0000", 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0624e-05 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069087+0000", 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0131999999999998e-05 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069168+0000", 2020-11-16T22:30:38.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3397000000000001e-05 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069181+0000", 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099632720000000004 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079144+0000", 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.366934256, 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001, 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.481 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012733930000000001 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078828+0000", 2020-11-16T22:30:38.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2857000000000002e-05 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078851+0000", 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9649e-05 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078881+0000", 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4674999999999999e-05 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078895+0000", 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9827000000000006e-05 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078985+0000", 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4489e-05 2020-11-16T22:30:38.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079000+0000", 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010194923 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.089195+0000", 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.488 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.360559911000003, 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:30:38.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:30:38.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:30:38.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:38.496 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.489904946999999, 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:38.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:30:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:30:38.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.2 117/109 [PushOp(4:584e1577:::10000000217.00000001:head, version: 86'1, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:584e1577:::10000000217.00000001:head@86'1, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:38.504 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.451716262000001, 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001, 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093718100000000004 2020-11-16T22:30:38.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993710+0000", 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3156000000000001e-05 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993723+0000", 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8623000000000001e-05 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993742+0000", 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:30:38.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993749+0000", 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9206e-05 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993829+0000", 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2907e-05 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993842+0000", 2020-11-16T22:30:38.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010134260000000001 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:08.003976+0000", 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.7051882129999996, 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:30:38.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:30:38.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:30:38.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:30:38.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:38.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:30:38.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:30:38.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:30:38.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:30:38.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:30:38.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:30:38.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:30:38.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:30:38.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:30:38.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:30:38.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:30:38.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:30:38.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:30:38.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1555551390000005, 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:38.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:38.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:30:38.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:30:38.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:30:38.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:30:38.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1554596989999997, 2020-11-16T22:30:38.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:30:38.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:30:38.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:30:38.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:30:38.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.555 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.1529253669999999, 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:38.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:30:38.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:30:38.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:30:38.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.565 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.8025055740000004, 2020-11-16T22:30:38.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:30:38.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:30:38.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.6947270449999996, 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:30:38.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:30:38.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:30:38.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:30:38.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:38.587 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.432819684, 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:38.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:30:38.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:30:38.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:30:38.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.598 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.4320854609999998, 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:38.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:30:38.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:30:38.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:30:38.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3702062220000002, 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:30:38.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:30:38.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:30:38.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:30:38.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:30:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.1001107369999996, 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:30:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:30:38.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:30:38.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:30:38.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19236 3.0 3:1e8917d7:::200.0000000e:head [write 2682968~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.2358248759999997, 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011138993, 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:38.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9647999999999994e-05 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:38.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208744+0000", 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6999999999999997e-06 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208753+0000", 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6699999999999999e-05 2020-11-16T22:30:38.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208769+0000", 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6299999999999998e-06 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208777+0000", 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4821000000000003e-05 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208822+0000", 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1468e-05 2020-11-16T22:30:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208903+0000", 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000170638 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.209074+0000", 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041805669999999996 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.213254+0000", 2020-11-16T22:30:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064380840000000002 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219692+0000", 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6050000000000005e-05 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219768+0000", 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4687000000000002e-05 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:38.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219803+0000", 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011138993 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:38.641 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:38.643 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:38.657+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:38.643 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:38.743 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:38.757+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:38.773 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:38.773 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:38.774 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:38.774 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:38.783 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:38.844 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:38.857+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:38.912 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:38.913 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:38.913 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:38.913 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:38.913 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:38.921 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:38.939 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:38.945 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:38.959+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:38.984 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:30:39.045 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:39.060+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:39.054 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:39.054 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 441.55926388699999, 2020-11-16T22:30:39.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:39.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:39.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:39.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:39.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:39.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:39.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.067 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 441.55914724399997, 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:39.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:39.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:39.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:39.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:39.079 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 441.55899887499999, 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:39.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:39.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:39.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:39.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:39.091 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.28990952800001, 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:39.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:39.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:39.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.099 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.81348283299999, 2020-11-16T22:30:39.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:39.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:39.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:39.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.108 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.74326036799999, 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.109 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:39.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:39.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:39.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:39.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.09332463699999, 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.120 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:39.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:39.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:39.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:39.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:39.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:39.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:39.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:39.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:39.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.130 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.087577329, 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.131 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:39.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:39.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:39.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:39.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.08644132500001, 2020-11-16T22:30:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.085294229, 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.079352857, 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.07700811800001, 2020-11-16T22:30:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.072261981, 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 141.553395438, 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:39.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:39.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:39.415 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.857571503, 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:39.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:39.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:39.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.423 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.840879301, 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:39.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:39.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.431 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 138.811693725, 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:39.432 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:39.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.439 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 30.979478986, 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:39.440 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:39.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:39.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.447 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18638 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 259@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 8.4255098499999992, 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997, 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0172e-05 2020-11-16T22:30:39.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642128+0000", 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1904e-05 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642140+0000", 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642155+0000", 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0720000000000002e-06 2020-11-16T22:30:39.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642162+0000", 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3480999999999997e-05 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642235+0000", 2020-11-16T22:30:39.654 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7229999999999999e-05 2020-11-16T22:30:39.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.654 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.654 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642293+0000", 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3079999999999997e-05 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642376+0000", 2020-11-16T22:30:39.655 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012180730000000001 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.643594+0000", 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040679215999999997 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.656 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684273+0000", 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9913000000000001e-05 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684323+0000", 2020-11-16T22:30:39.657 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5922999999999999e-05 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684349+0000", 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.658 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19235 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4.8590294099999998, 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619, 2020-11-16T22:30:39.659 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.660 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3933999999999997e-05 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208592+0000", 2020-11-16T22:30:39.661 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208600+0000", 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0356e-05 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.662 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208610+0000", 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3980000000000001e-06 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:39.663 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208614+0000", 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4449000000000001e-05 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208659+0000", 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6623e-05 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208756+0000", 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6582000000000006e-05 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208842+0000", 2020-11-16T22:30:39.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091079300000000002 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.209753+0000", 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096023089999999998 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219355+0000", 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.566e-05 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219401+0000", 2020-11-16T22:30:39.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6072000000000001e-05 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219457+0000", 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.668 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.669 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.669 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:39.669 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:39.669 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:39.671 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:39.671 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:39.671 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:39.671 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:39.671 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:39.672 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:39.684+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:39.672 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:39.771 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:39.784+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:39.814 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:39.815 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:39.824 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:39.871 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:39.885+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:39.963 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:39.963 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:39.971 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:30:39.971 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:30:39.971 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2020-11-16T22:30:39.971 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2020-11-16T22:30:39.973 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:39.987+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:39.974 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:30:40.074 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.087+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.146 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.146 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.156 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:40.175 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:40.189+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:40.276 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:40.290+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:40.302 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.302 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.313 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:40.377 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.390+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.478 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:40.492+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:40.489 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.489 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.499 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:40.581 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:40.596+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:40.679 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.692+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.686 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.686 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.696 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:40.780 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.793+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.825 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.825 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.834 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:40.881 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.894+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.968 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:40.968 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:40.978 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:40.981 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:40.995+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:40.992 INFO:tasks.ceph.mon.a.smithi138.stderr:2020-11-16T22:30:41.006+0000 7f9f33954700 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2020-11-16T22:30:41.003 INFO:teuthology.orchestra.run.smithi138.stderr:osd set-full-ratio 0.001 2020-11-16T22:30:41.082 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:41.096+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:41.109 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:41.109 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:41.118 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:41.183 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:41.197+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:41.255 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:41.255 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:41.267 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:41.284 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:41.298+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:41.385 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:41.399+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:41.419 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:41.419 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:41.430 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:41.485 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:41.499+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:41.579 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:41.579 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:41.586 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:41.600+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:41.589 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:41.672 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:41.687 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:41.701+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:41.737 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:41.737 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:41.748 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:41.788 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:41.802+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:41.819 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:41.840 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:41.860 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:41.862 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:41.862 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:41.862 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:41.887 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:41.887 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:30:41.887 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:30:41.887 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:30:41.888 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:41.890 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:41.904+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:41.897 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:41.991 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:42.005+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:42.012 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:30:42.015 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2020-11-16T22:30:42.016 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2020-11-16T22:30:42.037 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.050830572999999, 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:30:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:30:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.842680405999999, 2020-11-16T22:30:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:30:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:30:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:30:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:30:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.022329974999998, 2020-11-16T22:30:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001, 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.028568+0000", 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025738089999999998 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031142+0000", 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.242e-05 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031155+0000", 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6178e-05 2020-11-16T22:30:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031171+0000", 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1250000000000004e-06 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031178+0000", 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5498000000000002e-05 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031243+0000", 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1666e-05 2020-11-16T22:30:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.031255+0000", 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099647690000000001 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.041220+0000", 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012651465000000001 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.000874905000003, 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293, 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:30:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010396590000000001 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051063+0000", 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1077000000000002e-05 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051084+0000", 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6332000000000001e-05 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051111+0000", 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2245e-05 2020-11-16T22:30:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051123+0000", 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000182122 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051305+0000", 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2546e-05 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051318+0000", 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011890312 2020-11-16T22:30:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.063208+0000", 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.983246944999998, 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927, 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.067651+0000", 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001385074 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069037+0000", 2020-11-16T22:30:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9641999999999999e-05 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069056+0000", 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0559000000000001e-05 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069077+0000", 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0624e-05 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069087+0000", 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0131999999999998e-05 2020-11-16T22:30:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069168+0000", 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3397000000000001e-05 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.069181+0000", 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0099632720000000004 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079144+0000", 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0114927 2020-11-16T22:30:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.5 110/109 [PushOp(3:bec1251c:::200.00000007:head, version: 89'87, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bec1251c:::200.00000007:head@89'87, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.973343667999998, 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001, 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.077555+0000", 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012733930000000001 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078828+0000", 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2857000000000002e-05 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078851+0000", 2020-11-16T22:30:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9649e-05 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078881+0000", 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4674999999999999e-05 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078895+0000", 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9827000000000006e-05 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.078985+0000", 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4489e-05 2020-11-16T22:30:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.079000+0000", 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010194923 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.089195+0000", 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011639813000000001 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.966969323000001, 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:30:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:30:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:30:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.096314358999997, 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:30:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:30:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:30:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.2 117/109 [PushOp(4:584e1577:::10000000217.00000001:head, version: 86'1, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:584e1577:::10000000217.00000001:head@86'1, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.058125674000003, 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001, 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:30:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.992773+0000", 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093718100000000004 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993710+0000", 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3156000000000001e-05 2020-11-16T22:30:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993723+0000", 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8623000000000001e-05 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993742+0000", 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7500000000000003e-06 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993749+0000", 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9206e-05 2020-11-16T22:30:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993829+0000", 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2907e-05 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:07.993842+0000", 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010134260000000001 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:08.003976+0000", 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011203083000000001 2020-11-16T22:30:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.311597624999999, 2020-11-16T22:30:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:30:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:30:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:30:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.761964551, 2020-11-16T22:30:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:30:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:30:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.761869110999999, 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:30:42.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:30:42.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:30:42.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:42.347 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.759334779, 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:30:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:30:42.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:30:42.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:30:42.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.408914985999999, 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:42.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:30:42.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:30:42.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:30:42.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:30:42.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:42.367 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.301136457, 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:30:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:30:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:30:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:30:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.039229096, 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:30:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:30:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:30:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.038494872999999, 2020-11-16T22:30:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:30:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.9766156339999998, 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:30:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:30:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:30:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.7065201489999993, 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:30:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:30:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19236 3.0 3:1e8917d7:::200.0000000e:head [write 2682968~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.8422342880000002, 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011138993, 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208664+0000", 2020-11-16T22:30:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9647999999999994e-05 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208744+0000", 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6999999999999997e-06 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208753+0000", 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6699999999999999e-05 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208769+0000", 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6299999999999998e-06 2020-11-16T22:30:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208777+0000", 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4821000000000003e-05 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208822+0000", 2020-11-16T22:30:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1468e-05 2020-11-16T22:30:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:30:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.208903+0000", 2020-11-16T22:30:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000170638 2020-11-16T22:30:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.209074+0000", 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041805669999999996 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.213254+0000", 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064380840000000002 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219692+0000", 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6050000000000005e-05 2020-11-16T22:30:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219768+0000", 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4687000000000002e-05 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:30:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:34.219803+0000", 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011138993 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:42.834 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:42.836 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:42.850+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:42.836 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:42.936 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:42.950+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:42.967 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:42.967 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:42.968 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:42.968 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:42.977 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:43.019 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:30:43.061 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:43.075+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:43.137 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:43.152+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:43.146 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:43.146 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:43.147 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:43.147 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:43.147 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:43.156 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:43.238 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:43.253+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:43.329 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:43.329 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:43.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:43.329 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 445.83405140299999, 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.330 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:43.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:43.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:43.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:43.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:43.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:43.341 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 445.83393475999998, 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:43.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:43.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:43.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.352 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 445.83378639099999, 2020-11-16T22:30:43.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:43.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:43.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:43.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:43.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:43.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 148.56469704400001, 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.365 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:43.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:43.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:43.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 147.088270349, 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:43.373 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:43.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:43.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 147.018047884, 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:43.381 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.382 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:43.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:43.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:43.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:43.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:43.392 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.368112153, 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:43.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:43.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:43.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.400 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.362364845, 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:43.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:43.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:43.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:43.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:43.410 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.36122884100001, 2020-11-16T22:30:43.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:43.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:43.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:43.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:43.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:43.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.360081745, 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.422 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:43.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:43.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:43.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:43.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.432 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.35414037300001, 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.433 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:43.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:43.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:43.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.442 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.35179563400001, 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:43.443 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.444 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:43.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.347049497, 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:43.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:43.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:43.464 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 145.828182954, 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:43.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:43.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.13235901900001, 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:43.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:43.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:43.480 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.115666817, 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:43.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:43.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:43.488 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 143.086481241, 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:43.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:43.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:43.496 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 35.254266502, 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:43.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:43.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18638 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 259@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.700297365999999, 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997, 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:43.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0172e-05 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642128+0000", 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1904e-05 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642140+0000", 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642155+0000", 2020-11-16T22:30:43.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0720000000000002e-06 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642162+0000", 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3480999999999997e-05 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642235+0000", 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7229999999999999e-05 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642293+0000", 2020-11-16T22:30:43.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3079999999999997e-05 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642376+0000", 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012180730000000001 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.643594+0000", 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040679215999999997 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684273+0000", 2020-11-16T22:30:43.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9913000000000001e-05 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684323+0000", 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5922999999999999e-05 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684349+0000", 2020-11-16T22:30:43.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.714 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19235 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 9.1338169259999997, 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619, 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:43.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3933999999999997e-05 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208592+0000", 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208600+0000", 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0356e-05 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208610+0000", 2020-11-16T22:30:43.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3980000000000001e-06 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208614+0000", 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4449000000000001e-05 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.719 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208659+0000", 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6623e-05 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208756+0000", 2020-11-16T22:30:43.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6582000000000006e-05 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208842+0000", 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091079300000000002 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.209753+0000", 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096023089999999998 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219355+0000", 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.566e-05 2020-11-16T22:30:43.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219401+0000", 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6072000000000001e-05 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:43.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219457+0000", 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:43.724 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:43.726 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:43.740+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:43.726 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:43.827 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:43.841+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:43.867 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:43.867 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:43.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:34127 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 1368340~11190] snapc 1=[] ondisk+write+known_if_redirected e134)", 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:43.880290+0000", 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0020859390000000002, 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021041369999999998, 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:43.868 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 34127 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880290+0000", 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880290+0000", 2020-11-16T22:30:43.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1687000000000001e-05 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880312+0000", 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6720000000000001e-06 2020-11-16T22:30:43.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880317+0000", 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0984e-05 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880328+0000", 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8640000000000001e-06 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880334+0000", 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3891999999999999e-05 2020-11-16T22:30:43.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880378+0000", 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101437 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:43.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:43.880479+0000", 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000189536 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:43.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:30:43.876 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:43.877 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:43.878 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:43.927 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:43.941+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:osd set-full-ratio 0.95 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:44.025 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:30:44.026 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:44.028 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:44.042+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.029 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:44.038 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:44.051 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:44.072 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:44.073 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:44.073 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:44.073 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:44.129 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:44.142+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:44.173 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.173 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.183 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:44.229 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:44.244+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.329 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:44.344+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.335 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:44.431 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:44.446+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.479 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.479 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.491 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:44.534 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:44.549+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:44.624 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.624 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.632 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:44.646+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:44.635 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:44.733 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:44.747+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:44.764 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.764 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.774 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:44.834 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:44.847+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:44.935 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:44.948+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:44.943 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:44.943 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:44.952 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:45.035 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:30:45.036 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:45.050+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:45.080 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.080 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.090 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:45.136 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:45.151+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:45.237 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:45.251+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:45.245 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.245 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.257 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:45.338 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:45.352+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:45.400 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.401 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.410 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:45.439 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:45.453+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:45.540 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:45.554+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:45.549 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.550 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.560 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:45.641 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:45.655+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:45.702 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.702 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.712 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:45.742 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:45.755+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:45.843 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:45.857+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:45.857 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:45.857 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:45.869 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:45.944 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:45.958+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:46.007 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:46.007 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:46.017 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:30:46.045 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:46.058+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:46.084 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:46.145 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:46.159+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:46.154 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:46.154 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:46.163 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:46.233 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:46.246 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:46.261+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:46.254 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:46.274 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:46.275 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:46.275 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:46.275 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:46.335 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:46.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:46.335 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:46.335 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:46.335 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:46.345 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:46.387 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:46.401+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:46.515 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:46.516 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 449.02068131700003, 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:46.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:46.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:46.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:46.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:46.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:46.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:46.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:46.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:46.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 449.02056467400001, 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:46.528 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:46.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:46.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:46.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:46.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:46.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:46.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:46.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:46.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:46.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:46.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:46.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:46.942 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:46.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.942 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.942 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:46.942 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.943 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.944 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:46.944 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:46.946 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 449.02041630500003, 2020-11-16T22:30:46.947 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:46.947 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:46.947 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:46.947 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.948 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:46.949 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:46.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:46.950 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:46.950 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:47.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:47.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:47.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:47.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:47.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:47.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:47.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:47.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 151.75132695799999, 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:47.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:47.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:47.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:47.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:47.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.779 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 150.27490026300001, 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:47.780 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:47.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:47.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.787 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 150.20467779800001, 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:47.788 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.789 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:47.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:47.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:47.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:47.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:47.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:47.998 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:47.998 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:47.998 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:47.999 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.55474206700001, 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:48.000 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:48.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.54899475900001, 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:48.008 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.009 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:48.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.54785875499999, 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.220 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:48.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:48.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:48.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.229 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.54671165900001, 2020-11-16T22:30:48.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.231 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:48.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:48.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:48.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:48.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.441 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.54077028699999, 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:48.442 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.443 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:48.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.53842554799999, 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.453 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:48.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:48.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:48.463 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.53367941100001, 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:48.464 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:48.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:48.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:48.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:48.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.671 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:48.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:48.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:48.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.676 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 149.01481286800001, 2020-11-16T22:30:48.677 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.678 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:48.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:48.682 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.683 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.684 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 147.31898893300001, 2020-11-16T22:30:48.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:48.688 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:48.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.890 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 147.30229673100001, 2020-11-16T22:30:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1888 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'47, mlcod=88'45)", 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 146.27311115500001, 2020-11-16T22:30:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009, 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.255894+0000", 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090561799999999996 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256800+0000", 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3902e-05 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256814+0000", 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:30:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256823+0000", 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3090000000000003e-06 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256828+0000", 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4542000000000001e-05 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256873+0000", 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6295000000000002e-05 2020-11-16T22:30:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.256899+0000", 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.008460136 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265359+0000", 2020-11-16T22:30:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4038e-05 2020-11-16T22:30:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:49.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:20.265413+0000", 2020-11-16T22:30:49.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095193570000000009 2020-11-16T22:30:49.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.110 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 38.440896416000001, 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:49.111 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:49.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:49.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:49.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:49.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:49.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18638 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 259@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:49.118 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 15.88692728, 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997, 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0172e-05 2020-11-16T22:30:49.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642128+0000", 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1904e-05 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:49.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642140+0000", 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642155+0000", 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0720000000000002e-06 2020-11-16T22:30:49.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642162+0000", 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3480999999999997e-05 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:49.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642235+0000", 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7229999999999999e-05 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642293+0000", 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3079999999999997e-05 2020-11-16T22:30:49.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642376+0000", 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012180730000000001 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:49.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.643594+0000", 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040679215999999997 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684273+0000", 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9913000000000001e-05 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684323+0000", 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5922999999999999e-05 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:49.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684349+0000", 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.128 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19235 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:49.330 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:49.330 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.320446840000001, 2020-11-16T22:30:49.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619, 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:49.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3933999999999997e-05 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208592+0000", 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208600+0000", 2020-11-16T22:30:49.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0356e-05 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208610+0000", 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3980000000000001e-06 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208614+0000", 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4449000000000001e-05 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208659+0000", 2020-11-16T22:30:49.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6623e-05 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208756+0000", 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6582000000000006e-05 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208842+0000", 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091079300000000002 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.209753+0000", 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096023089999999998 2020-11-16T22:30:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219355+0000", 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.566e-05 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219401+0000", 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6072000000000001e-05 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219457+0000", 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619 2020-11-16T22:30:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:49.340 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:49.341 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:30:49.341 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:49.342 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:30:49.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:30:49.342 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.715510 2020-11-16T22:30:49.343 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.7155100083813044 2020-11-16T22:30:49.344 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:49.345 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:46.463+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:49.345 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:49.355+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:49.442 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:49.456+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:49.476 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.476 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:21812 3.0 e136/111 3:17051619:::200.0000000f:head v 136'7430, mlcod=136'7428)", 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:49.483838+0000", 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0072156599999999996, 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0072322100000000002, 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:49.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483838+0000", 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:49.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483838+0000", 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483875+0000", 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4159999999999998e-06 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483882+0000", 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.251e-06 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:49.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483892+0000", 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.95e-06 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483896+0000", 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7061999999999999e-05 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483934+0000", 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5669000000000001e-05 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:49.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.483959+0000", 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000121023 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:21813 3.0 e136/111 3:17051619:::200.0000000f:head v 136'7431, mlcod=136'7428)", 2020-11-16T22:30:49.483 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:49.489435+0000", 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0016189100000000001, 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001675527, 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489435+0000", 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489435+0000", 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4612e-05 2020-11-16T22:30:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489470+0000", 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6379999999999998e-06 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489476+0000", 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3219999999999999e-06 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489485+0000", 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0390000000000004e-06 2020-11-16T22:30:49.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489491+0000", 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3880000000000001e-05 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489524+0000", 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5043e-05 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:49.489539+0000", 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000104534 2020-11-16T22:30:49.489 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:30:49.490 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:49.491 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:49.501 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:49.522 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:49.542 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:49.543 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:49.557+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:49.544 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:49.637 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:49.643 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:49.658+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:49.744 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:49.759+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:49.783 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:49.845 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:49.859+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:49.926 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:49.926 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:49.937 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:49.946 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:49.960+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:50.044 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.1 primary-affinity to 0.71551 (8468912) 2020-11-16T22:30:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:30:50.047 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:50.062+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:50.117 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.118 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.129 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:50.173 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:50.187+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:50.249 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:50.263+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:50.266 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.266 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.277 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:50.350 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:50.365+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:50.411 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.412 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.421 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:50.451 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:50.465+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:50.548 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.549 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.551 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:50.566+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:50.559 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:50.652 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:50.667+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:50.731 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.731 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.740 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:50.753 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:50.768+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:50.854 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:50.869+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:50.880 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:50.881 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:50.892 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:50.955 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:50.969+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:51.040 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:51.040 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:51.052 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:51.055 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:51.070+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:51.157 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:51.171+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:51.237 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:51.237 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:51.247 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:51.291 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:51.305+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:51.357 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:51.371+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:51.382 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:51.382 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:51.392 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:51.457 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:51.471+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:51.555 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:51.558 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:51.572+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:51.568 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:51.569 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:51.578 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:51.659 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:51.674+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:51.702 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:51.715 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:51.715 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:51.723 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:51.726 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:51.744 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:51.745 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:51.745 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:51.745 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:51.768 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:51.783+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:51.858 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:51.859 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:51.859 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:30:51.859 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:51.861 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:51.875+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:51.868 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:51.962 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:51.976+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:52.038 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:52.038 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:30:52.038 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2020-11-16T22:30:52.038 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2020-11-16T22:30:52.038 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:52.047 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:52.059 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:30:52.063 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:52.077+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:52.164 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:52.178+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:52.181 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:30:52.181 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2020-11-16T22:30:52.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18 3.4 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 454.68630780199999, 2020-11-16T22:30:52.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001, 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508324+0000", 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7466000000000002e-05 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508381+0000", 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508393+0000", 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2040000000000003e-06 2020-11-16T22:30:52.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508403+0000", 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.578e-06 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508407+0000", 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0800999999999998e-05 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508448+0000", 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2249999999999997e-06 2020-11-16T22:30:52.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508454+0000", 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025931909999999999 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511047+0000", 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016627300000000001 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511214+0000", 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049334100000000001 2020-11-16T22:30:52.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.511707+0000", 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041840540000000004 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515891+0000", 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025952269999999999 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518486+0000", 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1851e-05 2020-11-16T22:30:52.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518558+0000", 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7510999999999999e-05 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518616+0000", 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010291872000000001 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.193 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19 3.4 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 454.68619115899997, 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001, 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508440+0000", 2020-11-16T22:30:52.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3362999999999997e-05 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508494+0000", 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4190000000000002e-06 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508502+0000", 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4416000000000001e-05 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508517+0000", 2020-11-16T22:30:52.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5970000000000003e-06 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508524+0000", 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1358e-05 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508556+0000", 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508561+0000", 2020-11-16T22:30:52.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044352990000000002 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.512996+0000", 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118819 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513115+0000", 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108686 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513224+0000", 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026838470000000001 2020-11-16T22:30:52.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.515908+0000", 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029318339999999999 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518840+0000", 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8198999999999997e-05 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518888+0000", 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.981e-05 2020-11-16T22:30:52.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518928+0000", 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010487176000000001 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:20 3.4 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force e16)", 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 454.68604278999999, 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034, 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.205 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508589+0000", 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2051000000000004e-05 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508671+0000", 2020-11-16T22:30:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3080000000000008e-06 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508679+0000", 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5747e-05 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508695+0000", 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4709999999999998e-06 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508700+0000", 2020-11-16T22:30:52.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508731+0000", 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2300000000000002e-06 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting_for_map not empty", 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.508735+0000", 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049086210000000002 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513644+0000", 2020-11-16T22:30:52.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120772 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.513765+0000", 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024179400000000001 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.514007+0000", 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020386839999999998 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.516045+0000", 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029287639999999999 2020-11-16T22:30:52.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.518974+0000", 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0906000000000002e-05 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519045+0000", 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9026999999999997e-05 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:23:17.519094+0000", 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010505034 2020-11-16T22:30:52.215 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:122 3.3 e87/82 3:ddb4e4d7:::200.00000002:head v 87'8, mlcod=85'6)", 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 157.41695344300001, 2020-11-16T22:30:52.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999, 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:52.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.777678+0000", 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013068719999999999 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778985+0000", 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2999999999999998e-06 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.778991+0000", 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1425e-05 2020-11-16T22:30:52.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779003+0000", 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.694e-06 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779008+0000", 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4316000000000001e-05 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779053+0000", 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5235000000000001e-05 2020-11-16T22:30:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.779078+0000", 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010318765000000001 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789397+0000", 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5611e-05 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:14.789452+0000", 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011774217999999999 2020-11-16T22:30:52.223 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:141 3.3 e88/82 3:d5c7a900:::200.00000003:head v 88'22, mlcod=88'20)", 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.940526748, 2020-11-16T22:30:52.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001, 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.254105+0000", 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029118550000000001 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257017+0000", 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.276e-06 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257022+0000", 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.551e-06 2020-11-16T22:30:52.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257031+0000", 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7299999999999996e-06 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257035+0000", 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2345999999999998e-05 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257068+0000", 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5916000000000001e-05 2020-11-16T22:30:52.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.257084+0000", 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022009067 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279093+0000", 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4377999999999999e-05 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.279157+0000", 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025052119000000001 2020-11-16T22:30:52.231 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4054 4.7 4:edb286fa:::1000000120b.00000000:head [write 0~7355] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.870304283, 2020-11-16T22:30:52.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005, 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4054 2020-11-16T22:30:52.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324327+0000", 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2478999999999999e-05 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324360+0000", 2020-11-16T22:30:52.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8949999999999997e-06 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324365+0000", 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0139000000000001e-05 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324375+0000", 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3700000000000003e-06 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324380+0000", 2020-11-16T22:30:52.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2839000000000001e-05 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324423+0000", 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010493900000000001 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324528+0000", 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012552099999999999 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.324654+0000", 2020-11-16T22:30:52.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067929700000000004 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.325333+0000", 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0084242999999999992 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333757+0000", 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8692e-05 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333816+0000", 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0975000000000003e-05 2020-11-16T22:30:52.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.333897+0000", 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095694460000000005 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.242 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:143 3.7 e88/82 3:f83d65d8:::200.00000004:head v 88'8, mlcod=16'6)", 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.220368552, 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997, 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.243 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.974263+0000", 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037627189999999999 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978026+0000", 2020-11-16T22:30:52.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1469999999999998e-06 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978032+0000", 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3160000000000003e-06 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978041+0000", 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1530000000000002e-06 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978047+0000", 2020-11-16T22:30:52.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2794999999999998e-05 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978110+0000", 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9142e-05 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.978139+0000", 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028190363 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006330+0000", 2020-11-16T22:30:52.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2921000000000003e-05 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006393+0000", 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.032129555999999997 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.250 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4619 4.7 4:f55188b9:::10000001440.00000000:head [write 0~3102] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.214621244, 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999, 2020-11-16T22:30:52.251 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4619 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980010+0000", 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5606999999999997e-05 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980076+0000", 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0799999999999995e-06 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980085+0000", 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3436000000000001e-05 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980109+0000", 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2705999999999999e-05 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980131+0000", 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5963999999999994e-05 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980197+0000", 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114999 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980312+0000", 2020-11-16T22:30:52.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118257 2020-11-16T22:30:52.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.980430+0000", 2020-11-16T22:30:52.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023544025 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003974+0000", 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067201800000000003 2020-11-16T22:30:52.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004646+0000", 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1453999999999997e-05 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004718+0000", 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013173300000000001 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004850+0000", 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024839278999999999 2020-11-16T22:30:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4620 4.7 4:ef65a013:::10000001441.00000000:head [write 0~2495] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.21348524000001, 2020-11-16T22:30:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001, 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4620 2020-11-16T22:30:52.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981146+0000", 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0801000000000003e-05 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981207+0000", 2020-11-16T22:30:52.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3010000000000007e-06 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981216+0000", 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7309e-05 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981233+0000", 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4110000000000004e-06 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981238+0000", 2020-11-16T22:30:52.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212303 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981451+0000", 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981554+0000", 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012547400000000001 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.981680+0000", 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022305854 2020-11-16T22:30:52.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003986+0000", 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091569899999999996 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004901+0000", 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5727000000000003e-05 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004967+0000", 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116433 2020-11-16T22:30:52.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005083+0000", 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023937031000000001 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4621 4.7 4:f21d2319:::10000001442.00000000:head [write 0~3058] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.212338144, 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001, 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.476 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4621 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982293+0000", 2020-11-16T22:30:52.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0463e-05 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982324+0000", 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7230000000000004e-06 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982329+0000", 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9719999999999998e-06 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982338+0000", 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.288e-06 2020-11-16T22:30:52.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.982343+0000", 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020147799999999999 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984358+0000", 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013939099999999999 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984497+0000", 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000146312 2020-11-16T22:30:52.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.984643+0000", 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019346253000000001 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003990+0000", 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001155058 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005145+0000", 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3369999999999997e-05 2020-11-16T22:30:52.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005218+0000", 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013705600000000001 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005355+0000", 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023061666000000001 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4626 4.7 4:ef0dac90:::10000001447.00000000:head [write 0~2670] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.20639677200001, 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787, 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4626 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988235+0000", 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000001e-05 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988265+0000", 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8810000000000004e-06 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988270+0000", 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9990000000000003e-06 2020-11-16T22:30:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988280+0000", 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6810000000000003e-06 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988285+0000", 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5209e-05 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988331+0000", 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106884 2020-11-16T22:30:52.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988437+0000", 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111507 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.988549+0000", 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015446404 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003995+0000", 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014163489999999999 2020-11-16T22:30:52.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005412+0000", 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6036000000000003e-05 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005478+0000", 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114877 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005593+0000", 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017357787 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.496 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4628 4.7 4:fb78df0d:::10000001449.00000000:head [write 0~1121] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.20405203300001, 2020-11-16T22:30:52.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287, 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4628 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.498 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990580+0000", 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1658e-05 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990621+0000", 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2020-11-16T22:30:52.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990629+0000", 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8810000000000008e-06 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990639+0000", 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2610000000000004e-06 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990643+0000", 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3643999999999998e-05 2020-11-16T22:30:52.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990677+0000", 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9504000000000001e-05 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990776+0000", 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016492699999999999 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.990941+0000", 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.013058134000000001 2020-11-16T22:30:52.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.003999+0000", 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016369139999999999 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005636+0000", 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1749000000000003e-05 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005678+0000", 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010443800000000001 2020-11-16T22:30:52.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005782+0000", 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01520287 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.4471.0:4632 4.7 4:e25d9a5d:::1000000144d.00000000:head [write 0~2585] snapc 1=[] ondisk+write+known_if_redirected e88)", 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 155.199305896, 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952, 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.508 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.4471", 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 4632 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995326+0000", 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9641e-05 2020-11-16T22:30:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995355+0000", 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1649999999999997e-06 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995361+0000", 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995371+0000", 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5059999999999997e-06 2020-11-16T22:30:52.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995376+0000", 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8427000000000003e-05 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995414+0000", 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8239000000000006e-05 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995512+0000", 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104553 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1", 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:16.995617+0000", 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083875359999999993 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.004004+0000", 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018287220000000001 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005833+0000", 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0164999999999997e-05 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.005893+0000", 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119548 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.006013+0000", 2020-11-16T22:30:52.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010686952 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:145 3.2 e88/82 3:5e41ab69:::200.00000005:head v 88'2, mlcod=0'0)", 2020-11-16T22:30:52.518 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 154.680439353, 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999, 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.514192+0000", 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023923260000000002 2020-11-16T22:30:52.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516585+0000", 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.712e-06 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516592+0000", 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9380000000000004e-06 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516602+0000", 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.366e-06 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516608+0000", 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4353e-05 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516652+0000", 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9009e-05 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.516671+0000", 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017171191999999998 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533842+0000", 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9325e-05 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:17.533892+0000", 2020-11-16T22:30:52.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019699220999999999 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1100 3.6 e88/82 3:7bf7e7b5:::200.00000006:head v 88'13, mlcod=87'11)", 2020-11-16T22:30:52.526 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.984615418, 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543, 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.210016+0000", 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012143550000000001 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211231+0000", 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6150000000000003e-06 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211240+0000", 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211254+0000", 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2439999999999999e-06 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211263+0000", 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4832000000000003e-05 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211358+0000", 2020-11-16T22:30:52.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9218000000000001e-05 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.211377+0000", 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093729750000000004 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220750+0000", 2020-11-16T22:30:52.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1431000000000002e-05 2020-11-16T22:30:52.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.220812+0000", 2020-11-16T22:30:52.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.01079543 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(mds.0.3:1102 3.6 e88/82 3:654134d2:::mds0_openfiles.0:head v 88'14, mlcod=87'11)", 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:52.735 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.967923216, 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234, 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226708+0000", 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120457 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226829+0000", 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0276e-05 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226839+0000", 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4113e-05 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226853+0000", 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3320000000000003e-06 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226861+0000", 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6007999999999995e-05 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226937+0000", 2020-11-16T22:30:52.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2898999999999999e-05 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.226960+0000", 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099509460000000004 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236910+0000", 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7202999999999998e-05 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:28:19.236998+0000", 2020-11-16T22:30:52.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010289234 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(3.6 118/117 [PushOp(3:7bf7e7b5:::200.00000006:head, version: 88'83, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7bf7e7b5:::200.00000006:head@88'83, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:30:52.743 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 44.106522900999998, 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002, 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.088109+0000", 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010458380000000001 2020-11-16T22:30:52.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089155+0000", 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5072000000000001e-05 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089170+0000", 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7062e-05 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089187+0000", 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9990000000000008e-06 2020-11-16T22:30:52.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089195+0000", 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6908000000000003e-05 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089272+0000", 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4375e-05 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.089286+0000", 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083701639999999994 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:08.097656+0000", 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0095474180000000002 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:18638 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 259@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.552553764999999, 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997, 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.751 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642078+0000", 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0172e-05 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642128+0000", 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1904e-05 2020-11-16T22:30:52.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642140+0000", 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642155+0000", 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0720000000000002e-06 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642162+0000", 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3480999999999997e-05 2020-11-16T22:30:52.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642235+0000", 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7229999999999999e-05 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642293+0000", 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3079999999999997e-05 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.642376+0000", 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012180730000000001 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.643594+0000", 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040679215999999997 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684273+0000", 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9913000000000001e-05 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684323+0000", 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5922999999999999e-05 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:30.684349+0000", 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042271024999999997 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:19235 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 17.986073325, 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619, 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.761 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208558+0000", 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3933999999999997e-05 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208592+0000", 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4429999999999997e-06 2020-11-16T22:30:52.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208600+0000", 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0356e-05 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208610+0000", 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3980000000000001e-06 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208614+0000", 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4449000000000001e-05 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208659+0000", 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6623e-05 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208756+0000", 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6582000000000006e-05 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.208842+0000", 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00091079300000000002 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.209753+0000", 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096023089999999998 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219355+0000", 2020-11-16T22:30:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.566e-05 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219401+0000", 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6072000000000001e-05 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:34.219457+0000", 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010898619 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(mds.0.3:21322 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 695@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e136)", 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2020-11-16T22:30:46.500399+0000", 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 5.6942328120000001, 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.070656479999999994, 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2020-11-16T22:30:52.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500399+0000", 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2020-11-16T22:30:52.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500399+0000", 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6279999999999999e-05 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500425+0000", 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3789999999999997e-06 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500430+0000", 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8479999999999996e-06 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500440+0000", 2020-11-16T22:30:52.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2919999999999997e-06 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500445+0000", 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3235e-05 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500508+0000", 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3792999999999998e-05 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500562+0000", 2020-11-16T22:30:52.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4068999999999997e-05 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.500656+0000", 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069608278999999995 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.570264+0000", 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00067750099999999995 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.570942+0000", 2020-11-16T22:30:52.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9073000000000003e-05 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.570991+0000", 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4731000000000001e-05 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2020-11-16T22:30:52.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2020-11-16T22:30:46.571055+0000", 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.070656479999999994 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2020-11-16T22:30:52.780 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:30:52.782 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:52.782 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:52.796+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:52.882 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:52.897+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:52.916 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:52.916 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:52.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.916 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:22379 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 863@2509130] snapc 1=[] ondisk+write+known_if_redirected+full_force e138)", 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:52.929956+0000", 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00047927000000000001, 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00050571700000000004, 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:30:52.917 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.929956+0000", 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.929956+0000", 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4773000000000002e-05 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.929981+0000", 2020-11-16T22:30:52.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0240000000000003e-06 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.929987+0000", 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1586999999999999e-05 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.929999+0000", 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.403e-06 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930003+0000", 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5012000000000002e-05 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930048+0000", 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000108101 2020-11-16T22:30:52.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930156+0000", 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000160602 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930317+0000", 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00036050200000000001 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:36660 4.1 4.c30f5c31 (undecoded) ondisk+read+known_if_redirected e138)", 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:52.930079+0000", 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00035680199999999998, 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00043120799999999998, 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:30:52.925 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 36660 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930079+0000", 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930079+0000", 2020-11-16T22:30:52.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9174e-05 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930098+0000", 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8840000000000004e-06 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930102+0000", 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0480000000000001e-06 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930109+0000", 2020-11-16T22:30:52.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.382e-06 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930112+0000", 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3488000000000002e-05 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.930 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:36661 4.1 4.c30f5c31 (undecoded) ondisk+read+known_if_redirected e138)", 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:52.930136+0000", 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00029977999999999998, 2020-11-16T22:30:52.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000407205, 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 36661 2020-11-16T22:30:52.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930136+0000", 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930136+0000", 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8885e-05 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930155+0000", 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1659999999999998e-06 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930158+0000", 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9139999999999996e-06 2020-11-16T22:30:52.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930164+0000", 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5689999999999999e-06 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930166+0000", 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0534000000000003e-05 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.937 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:22381 4.5 e138/117 4:a611fa68:::1000000188a.00000002:head v 138'2016, mlcod=138'2014)", 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:52.930314+0000", 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.000121997, 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00027279300000000002, 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:52.938 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930314+0000", 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:52.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930314+0000", 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2654999999999998e-05 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930346+0000", 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7239999999999997e-06 2020-11-16T22:30:52.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930352+0000", 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0689e-05 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930363+0000", 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9439999999999996e-06 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930368+0000", 2020-11-16T22:30:52.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7246999999999998e-05 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930405+0000", 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5362e-05 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:52.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:52.930420+0000", 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010662099999999999 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.944 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:52.945 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:30:52.945 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 4 2020-11-16T22:30:52.945 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:52.946 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:52.986 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:53.001+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:53.070 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:30:53.085 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:53.099+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:53.118 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:53.118 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:30:53.119 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:30:53.119 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:30:53.119 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:53.128 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:53.185 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:53.199+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:53.276 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:53.277 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 167.96867466699999, 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:53.278 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:30:53.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:30:53.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:30:53.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.286 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 162.96817177899999, 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.287 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:30:53.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:30:53.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:30:53.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:30:53.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.296 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 157.03950054699999, 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:30:53.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:30:53.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:30:53.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:30:53.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.306 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 156.32221407599999, 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:30:53.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:30:53.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:30:53.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:30:53.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.316 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07805186300001, 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:30:53.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:30:53.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:30:53.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:30:53.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1106 3.6 3:7bf7e7b5:::200.00000006:head [write 1368034~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07722507899999, 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999, 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212026+0000", 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9792000000000001e-05 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212066+0000", 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7160000000000002e-06 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212071+0000", 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9000000000000006e-06 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212078+0000", 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5749999999999999e-06 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212081+0000", 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019514948000000001 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231596+0000", 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6363000000000002e-05 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231652+0000", 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8246999999999996e-05 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.231751+0000", 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024807480000000001 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234231+0000", 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044419059999999998 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238673+0000", 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1994999999999999e-05 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238725+0000", 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7255999999999994e-05 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238802+0000", 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026776445999999999 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.077142791, 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.337 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:30:53.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:30:53.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:30:53.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:30:53.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07706044700001, 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.347 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:30:53.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:30:53.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:30:53.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:30:53.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07698532699999, 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.357 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:30:53.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:30:53.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:30:53.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:30:53.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.076906393, 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.367 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:30:53.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:30:53.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:30:53.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:30:53.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.376 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.076828971, 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:30:53.377 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:30:53.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.386 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07674880900001, 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:30:53.387 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:30:53.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:30:53.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.396 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.076666996, 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:30:53.397 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:30:53.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:30:53.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:30:53.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:30:53.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07657361, 2020-11-16T22:30:53.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:53.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:30:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:30:53.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:30:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:30:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.076453214, 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.621 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:30:53.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:30:53.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:30:53.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:30:53.629 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.630 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 154.07635314699999, 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.631 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.632 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:30:53.633 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:30:53.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:30:53.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.638 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:30:53.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.640 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 112.67608016299999, 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.641 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:30:53.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:30:53.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:30:53.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:30:53.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.830107229999996, 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:30:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:30:53.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:30:53.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:30:53.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 54.379305088999999, 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.660 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:30:53.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:30:53.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:30:53.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:30:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:53.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 54.080928030000003, 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:53.670 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:30:53.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:30:53.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:30:53.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:30:53.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:30:53.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:53.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:30:53.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:53.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.883 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:53.884 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:53.884 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:30:53.884 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:30:53.886 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:53.900+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:53.886 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:30:53.986 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:54.000+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:54.032 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:30:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:30:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(4.0 epoch 139/117 rep_tid 2846 entries 139'2933 (0'0) error 4:caefcac0:::1000000188a.00000001:head by client.4471.0:37005 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 139'2931)", 2020-11-16T22:30:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:54.046762+0000", 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00017714599999999999, 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020641799999999999, 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046762+0000", 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:30:54.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046762+0000", 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6361000000000001e-05 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046788+0000", 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3690000000000007e-06 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046797+0000", 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4901e-05 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:30:54.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046811+0000", 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1e-06 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046818+0000", 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9796000000000003e-05 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:30:54.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:54.046857+0000", 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5526999999999995e-05 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:54.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:30:54.039 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:30:54.039 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:30:54.039 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:30:54.039 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:54.042 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:30:54.054 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:54.077 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:54.078 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T22:30:54.078 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:30:54.078 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:30:54.086 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:54.101+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:54.188 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:54.202+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:54.195 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:54.195 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:54.206 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:30:54.288 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:54.303+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:54.387 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:54.387 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:54.389 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:54.403+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:54.397 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:30:54.489 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:54.504+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:54.528 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:54.528 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:54.538 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T22:30:54.590 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:54.605+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:54.691 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:54.705+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:54.709 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:54.709 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:54.719 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T22:30:54.792 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:54.807+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:54.887 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:54.887 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:54.893 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:54.907+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:54.896 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:54.994 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:30:55.008+0000 7ff603c93700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20750) UID: 0 2020-11-16T22:30:55.041 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:55.042 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:55.052 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:30:55.057 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2020-11-16T22:30:55.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:30:55.058 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2] 2020-11-16T22:30:55.058 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.0 doing powercycle of ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:30:55.058 INFO:teuthology.orchestra.console:Power off smithi173 2020-11-16T22:30:55.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T22:30:59.260 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:30:59.261 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:30:59.263 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:59.263 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:59.264 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:30:59.264 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:59.276+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:59.274 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:30:59.363 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:59.377+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:59.410 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:30:59.410 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:30:59.415 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0: {} 2020-11-16T22:30:59.421 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:30:59.437 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:30:59.458 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:30:59.459 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T22:30:59.459 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:30:59.459 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:30:59.464 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:30:59.478+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:30:59.565 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:30:59.579+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:30:59.611 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:23530 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 1057@620331] snapc 1=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:59.624580+0000", 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001685621, 2020-11-16T22:30:59.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0017064020000000001, 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624580+0000", 2020-11-16T22:30:59.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624580+0000", 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6049000000000001e-05 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624606+0000", 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3809999999999998e-06 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:59.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624611+0000", 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.859e-06 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624621+0000", 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2930000000000002e-06 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624625+0000", 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1126e-05 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624657+0000", 2020-11-16T22:30:59.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0630000000000001e-05 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624717+0000", 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4783000000000004e-05 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624812+0000", 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00023212100000000001 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:38557 4.1 4:8c3af0c3:::1000000188a.00000000:head [read 565248~55083 [1057@620331]] snapc 0=[] ondisk+read+known_if_redirected e139)", 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:59.624965+0000", 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001300205, 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0013772540000000001, 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "delayed", 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:30:59.620 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 38557 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:59.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624965+0000", 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624965+0000", 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2160000000000001e-05 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624987+0000", 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0210000000000001e-06 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624991+0000", 2020-11-16T22:30:59.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5989999999999996e-06 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.624999+0000", 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9570000000000004e-06 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625003+0000", 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001271115 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.626274+0000", 2020-11-16T22:30:59.625 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7367999999999997e-05 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for rw locks", 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.626331+0000", 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00136622 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:59.626 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23532 4.5 e139/117 4:a611fa68:::1000000188a.00000002:head v 139'2234, mlcod=139'2232)", 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:59.625044+0000", 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0012217479999999999, 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0013454809999999999, 2020-11-16T22:30:59.627 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625044+0000", 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.628 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.629 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:30:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625044+0000", 2020-11-16T22:30:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2141999999999998e-05 2020-11-16T22:30:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625076+0000", 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9459999999999997e-06 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625081+0000", 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.968e-06 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625088+0000", 2020-11-16T22:30:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1360000000000001e-06 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625091+0000", 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8065999999999997e-05 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625139+0000", 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1758000000000003e-05 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:30:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625171+0000", 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000127016 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:30:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.038 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:38558 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 451918~14909] snapc 1=[] ondisk+write+known_if_redirected e139)", 2020-11-16T22:31:00.038 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:30:59.625257+0000", 2020-11-16T22:31:00.038 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0010089579999999999, 2020-11-16T22:31:00.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011711569999999999, 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 38558 2020-11-16T22:31:00.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625257+0000", 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625257+0000", 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3822000000000002e-05 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:00.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625290+0000", 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2809999999999997e-06 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625297+0000", 2020-11-16T22:31:00.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3885e-05 2020-11-16T22:31:00.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:00.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625311+0000", 2020-11-16T22:31:00.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0760000000000002e-06 2020-11-16T22:31:00.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.625316+0000", 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001042252 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:00.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:59.626358+0000", 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001101316 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:00.248 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 4 2020-11-16T22:31:00.249 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:00.250 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:31:00.251 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:00.251 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:00.264+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:00.351 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:00.365+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:00.432 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:00.433 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:00.433 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:00.433 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:00.433 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:00.442 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:00.451 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:00.466+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:00.552 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:00.566+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:00.579 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:00.580 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 175.27185413999999, 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:00.581 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:00.582 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.583 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:00.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:00.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:00.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:00.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:00.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:00.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:00.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:00.790 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:00.791 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:00.791 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:00.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.791 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:00.994 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:00.994 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 170.27135125199999, 2020-11-16T22:31:00.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:00.994 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.995 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:00.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:00.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:00.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:00.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:00.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:01.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:01.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:01.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:01.206 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:01.207 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:01.207 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:01.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.207 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 164.34268001999999, 2020-11-16T22:31:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:01.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:01.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:01.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:01.623 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 163.62539354899999, 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:01.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:01.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:01.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:01.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:02.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1102 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:31:02.039 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.38123133600001, 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999, 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:02.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211199+0000", 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025104599999999999 2020-11-16T22:31:02.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211450+0000", 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5520000000000001e-06 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211457+0000", 2020-11-16T22:31:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2707e-05 2020-11-16T22:31:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:02.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211470+0000", 2020-11-16T22:31:02.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3130000000000001e-06 2020-11-16T22:31:02.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.211477+0000", 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065712670000000004 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:02.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218048+0000", 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000101609 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.218150+0000", 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0082785040000000008 2020-11-16T22:31:02.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.226428+0000", 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00776065 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:02.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234189+0000", 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037752689999999999 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237964+0000", 2020-11-16T22:31:02.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3607999999999999e-05 2020-11-16T22:31:02.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:02.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238018+0000", 2020-11-16T22:31:02.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1122e-05 2020-11-16T22:31:02.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238099+0000", 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026899646999999999 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1107 3.6 3:7bf7e7b5:::200.00000006:head [write 1369420~2213 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.380322264, 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387, 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:02.456 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212108+0000", 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9561999999999998e-05 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:02.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212148+0000", 2020-11-16T22:31:02.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4209999999999997e-06 2020-11-16T22:31:02.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:02.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212152+0000", 2020-11-16T22:31:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9509999999999995e-06 2020-11-16T22:31:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212161+0000", 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0290000000000001e-06 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212164+0000", 2020-11-16T22:31:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.019941815000000002 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232106+0000", 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0988000000000005e-05 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232187+0000", 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011277699999999999 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232300+0000", 2020-11-16T22:31:02.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003030222 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235330+0000", 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035172929999999999 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:02.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238847+0000", 2020-11-16T22:31:02.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4656e-05 2020-11-16T22:31:02.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238902+0000", 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4672999999999994e-05 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238987+0000", 2020-11-16T22:31:02.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026878387 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:02.872 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.38023992000001, 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:02.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:02.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:03.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:03.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:03.288 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.38016479999999, 2020-11-16T22:31:03.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:03.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:03.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:03.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:03.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:03.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:03.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:03.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:03.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:03.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:03.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:03.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.380085866, 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:03.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:03.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:03.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:03.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:03.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:03.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:03.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:03.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:04.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:04.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:04.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:04.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:04.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:04.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.122 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.380008444, 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:04.123 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.539 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:04.539 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.37992828200001, 2020-11-16T22:31:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.158 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.379846469, 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:05.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:05.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:05.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.379753083, 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:05.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.379632687, 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:05.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:05.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:05.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:05.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:05.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:05.393 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.37953261999999, 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:05.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:05.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:05.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:05.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 119.97925963599999, 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.404 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:05.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:05.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:05.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:05.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:05.414 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 84.133286702999996, 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:05.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:05.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:05.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:05.423 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.682484561999999, 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:05.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:05.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:05.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:05.432 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:05.433 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.384107503000003, 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:05.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:05.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:05.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:05.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:05.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:05.444 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:05.445 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.50715948899999996, 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:05.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:05.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:05.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:05.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:05.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:05.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:05.458 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:05.459 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:05.460 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:31:05.460 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:31:05.472 INFO:tasks.workunit.client.0.smithi138.stdout:20000 mapwrite 0x840504 thru 0x847a17 (0x7514 bytes) 2020-11-16T22:31:05.473 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:05.474 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:05.474+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:05.560 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:05.574+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:05.612 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:05.612 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:40141 4.1 e142/129 4:8c3af0c3:::1000000188a.00000000:head v 142'3572, mlcod=142'3570)", 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.623768+0000", 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0036751129999999998, 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036910599999999999, 2020-11-16T22:31:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623768+0000", 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:05.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623768+0000", 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0457999999999993e-05 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623859+0000", 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3950000000000007e-06 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623868+0000", 2020-11-16T22:31:05.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4960000000000001e-05 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623883+0000", 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7129999999999999e-06 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623890+0000", 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4190999999999998e-05 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:05.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623934+0000", 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5077000000000001e-05 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.623959+0000", 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019079399999999999 2020-11-16T22:31:05.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24384 3.5 3:b02ffd62:::200.00000010:head [write 915814~1922 [fadvise_dontneed] in=1922b] snapc 0=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.626713+0000", 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000730161, 2020-11-16T22:31:05.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00078781599999999999, 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626713+0000", 2020-11-16T22:31:05.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626713+0000", 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8501000000000003e-05 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626752+0000", 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4670000000000001e-06 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:05.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626758+0000", 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0784999999999999e-05 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626769+0000", 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8289999999999998e-06 2020-11-16T22:31:05.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626774+0000", 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1918000000000001e-05 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:05.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626816+0000", 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010299899999999999 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.626919+0000", 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020549899999999999 2020-11-16T22:31:05.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:31:05.627 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:05.628 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:05.629 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:05.646 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:05.648 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:05.649 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:05.661 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:05.675+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:05.762 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:05.776+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:05.764 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:05.764 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:05.774 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:05.862 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:05.877+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:05.913 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:05.913 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:05.925 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:05.966 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:05.980+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:06.064 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.079+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:06.065 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.065 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.076 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:06.109 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:31:06.165 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:06.179+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:06.253 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.253 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.264 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:06.265 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.280+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:06.366 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:06.381+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:06.401 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.401 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.411 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:06.467 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.482+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:06.557 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.557 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.567 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:06.569 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:06.583+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:06.670 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.684+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:06.707 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.708 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.717 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:06.771 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.785+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:06.859 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:06.859 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:06.869 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:06.886 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:06.886+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:06.973 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:06.987+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:07.000 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:07.001 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:07.011 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:07.073 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.088+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.174 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:07.189+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:07.182 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:07.182 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:07.193 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:07.275 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.290+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.375 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:07.375 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:07.377 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:07.391+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:07.386 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:07.478 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.492+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.530 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:07.530 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:07.541 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:07.579 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:07.593+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:07.649 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:07.679 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.694+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.719 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:07.719 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:07.729 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:07.800 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:07.812 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.827+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.822 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:07.825 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:07.825 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:07.881 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:07.895+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:07.915 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:07.916 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 69.928633746000003, 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:07.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:07.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:07.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:07.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:07.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:07.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:07.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:07.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:07.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:07.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:07.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:07.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:07.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:07.927 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.927 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.928 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.928 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:07.928 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:07.929 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.720483579000003, 2020-11-16T22:31:07.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:07.929 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.929 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:07.929 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:07.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:07.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:07.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:07.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:07.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:07.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:07.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:07.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:07.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:07.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:07.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:07.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:07.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:07.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:07.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:07.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:07.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:07.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:07.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:07.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:07.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:07.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:07.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:07.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:07.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:07.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:07.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:07.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:07.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:07.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:07.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:07.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:07.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:07.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:07.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:07.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:07.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.948 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.878678077999993, 2020-11-16T22:31:07.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293, 2020-11-16T22:31:07.950 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.950 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:07.950 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:31:07.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.050024+0000", 2020-11-16T22:31:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010396590000000001 2020-11-16T22:31:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051063+0000", 2020-11-16T22:31:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1077000000000002e-05 2020-11-16T22:31:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051084+0000", 2020-11-16T22:31:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6332000000000001e-05 2020-11-16T22:31:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051111+0000", 2020-11-16T22:31:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2245e-05 2020-11-16T22:31:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051123+0000", 2020-11-16T22:31:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000182122 2020-11-16T22:31:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051305+0000", 2020-11-16T22:31:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2546e-05 2020-11-16T22:31:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.051318+0000", 2020-11-16T22:31:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011890312 2020-11-16T22:31:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.063208+0000", 2020-11-16T22:31:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013184293 2020-11-16T22:31:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.844772496000004, 2020-11-16T22:31:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:07.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:07.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:07.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:07.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:07.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:07.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:07.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:07.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:07.970 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.970 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.970 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.971 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:07.971 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:07.971 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.974117531999994, 2020-11-16T22:31:07.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:07.972 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.972 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:07.972 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:07.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:07.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:07.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:07.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:07.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:07.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:07.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:07.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:07.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:07.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:07.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:07.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:07.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:07.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.189400798000001, 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:07.982 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:07.983 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:07.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:07.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:07.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:07.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:07.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:07.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:07.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:07.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:07.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:07.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:07.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:07.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:07.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:07.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:07.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:07.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:07.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:07.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:07.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:07.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:07.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:07.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:07.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:07.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:07.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:07.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:07.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:07.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:07.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:07.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:07.996 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.996 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:07.996 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:07.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.997 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:07.997 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:07.997 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.639767724000002, 2020-11-16T22:31:07.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:07.997 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.639672284, 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.009 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:08.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:08.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:08.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:08.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.637137952000003, 2020-11-16T22:31:08.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:08.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.286718159000003, 2020-11-16T22:31:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.178939630000002, 2020-11-16T22:31:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:31:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:31:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:31:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:31:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.917032269000003, 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.916298046000001, 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.854418807000002, 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:08.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:08.288 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.584323322000003, 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.8438124419999999, 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3848659840000002, 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:08.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:08.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:08.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:08.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:08.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:08.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.7027950029999999, 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:08.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:08.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.70234913, 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.533 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:08.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:08.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 2.791152071, 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:08.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:08.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:08.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:08.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:08.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:08.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:08.551 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:08.551 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:31:08.553 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:08.566+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:08.553 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:08.652 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:08.666+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:08.692 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:08.692 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:08.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.692 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:41338 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1773568~49152 [1310@4194304] in=49152b] snapc 1=[] ondisk+write+known_if_redirected e143)", 2020-11-16T22:31:08.692 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:08.706366+0000", 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.000548917, 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00057394499999999997, 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:08.693 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 41338 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706366+0000", 2020-11-16T22:31:08.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706366+0000", 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1527999999999999e-05 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706407+0000", 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6789999999999998e-06 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706411+0000", 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.798e-06 2020-11-16T22:31:08.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706418+0000", 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8959999999999999e-06 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706421+0000", 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018990399999999999 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706611+0000", 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5558e-05 2020-11-16T22:31:08.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706656+0000", 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2293999999999994e-05 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706748+0000", 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00038265700000000001 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.700 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:41339 4.1 4.c30f5c31 (undecoded) ondisk+read+known_if_redirected e143)", 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:08.706699+0000", 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.000215567, 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00031868499999999998, 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:08.701 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 41339 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706699+0000", 2020-11-16T22:31:08.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706699+0000", 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9219999999999999e-05 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706718+0000", 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9809999999999997e-06 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706722+0000", 2020-11-16T22:31:08.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1667e-05 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706734+0000", 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.456e-06 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:08.706737+0000", 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8324000000000001e-05 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.706 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:08.707 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:31:08.707 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:08.707 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:08.753 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:08.767+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:08.842 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:08.842 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:08.842 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:08.842 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:08.842 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:08.852 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:08.854 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:08.868+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:08.954 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:08.969+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:08.989 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 183.6820409, 2020-11-16T22:31:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 178.681538012, 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 172.75286678000001, 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 172.03558030900001, 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.79042668, 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.79035156, 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.79027262599999, 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.79019520400001, 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.790115042, 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:09.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:09.486 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.486 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:09.487 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.79003322899999, 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:09.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:09.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:09.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:09.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:09.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:09.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:09.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:09.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:09.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:09.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:09.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:09.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.78993984300001, 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:09.904 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:09.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:09.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:10.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:10.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:10.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:10.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:10.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:10.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:10.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:10.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:10.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:10.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:10.320 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.78981944700001, 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:10.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:10.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:10.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:10.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:10.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:10.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:10.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:10.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:10.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:10.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:10.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 169.78971938000001, 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:10.737 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:10.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:10.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:10.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:10.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:10.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:10.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:10.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:10.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:10.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:10.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:10.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:10.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:10.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:10.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:10.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:10.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:11.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:11.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.38944639600001, 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:11.154 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:11.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:11.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:11.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:11.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:11.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:11.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 92.543473462999998, 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:11.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:11.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:11.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:11.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:11.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:11.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:11.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:11.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:11.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:11.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 70.092671322000001, 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:11.986 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:11.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:12.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:12.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:12.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:12.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:12.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:12.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:12.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:12.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:12.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:12.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:12.401 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.794294262999998, 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:12.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:12.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:12.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:12.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:12.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:12.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:12.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:12.614 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.615 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:12.818 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:12.818 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 8.9173462489999995, 2020-11-16T22:31:12.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:12.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:12.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:12.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:12.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:12.826 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.8647482630000001, 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:12.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:12.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:12.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:13.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:13.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:13.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:13.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:13.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:13.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:13.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:13.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:13.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:13.239 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 3.8637049370000001, 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:13.240 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:13.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:13.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:13.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:13.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:13.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:13.248 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:13.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:13.248 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:13.450 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:13.450 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:13.451 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:31:17.612 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:17.615 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:31:17.616 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:17.617 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:13.467+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:17.618 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:17.632+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:17.719 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:17.734+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:17.758 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:17.758 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:17.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44927 4.5 4:a611fa68:::1000000188a.00000002:head [zero 1777319~62510] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:17.758 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:17.769472+0000", 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0031179689999999999, 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003136977, 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:17.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44927 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769472+0000", 2020-11-16T22:31:17.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769472+0000", 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0825000000000001e-05 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769492+0000", 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9370000000000003e-06 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769497+0000", 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.03e-05 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769508+0000", 2020-11-16T22:31:17.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4550000000000003e-06 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769513+0000", 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6300999999999997e-05 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769549+0000", 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1428000000000006e-05 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:17.769641+0000", 2020-11-16T22:31:17.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016924600000000001 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:17.766 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:31:17.970 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:17.970 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2020-11-16T22:31:17.972 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:17.973 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:17.973 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:17.973 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:17.973 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:17.973 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:17.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:17.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:17.974 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:17.975 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:17.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:18.072 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.086+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.112 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.112 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.122 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:18.204 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.219+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.296 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.296 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.305 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:18.306 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.288+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.374 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.388+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.459 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.460 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.470 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:18.514 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:18.529+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:18.576 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.590+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.577 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2020-11-16T22:31:18.609 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.610 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.620 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:18.676 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.690+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.777 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:18.791+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:18.803 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.803 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.813 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:18.878 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:18.892+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:18.949 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:18.949 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:18.959 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:18.979 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:18.993+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:19.080 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.094+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.133 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:19.133 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:19.143 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:19.184 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.198+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.282 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.296+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.323 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:19.323 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:19.333 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:19.382 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:19.397+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:19.480 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:19.480 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:19.484 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.498+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.492 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:19.584 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.598+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.617 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:31:19.680 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:19.680 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:19.685 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:19.700+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:19.692 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:19.787 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:19.801+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:19.868 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:19.869 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:19.878 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:19.887 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:19.902+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:19.975 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:19.988 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:20.003+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:20.018 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:20.018 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:20.028 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:20.089 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:20.103+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:20.123 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:20.145 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:20.145 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:20.145 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:20.145 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:20.146 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:20.147 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:20.148 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:20.168 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:20.169 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:20.178 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:20.219 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:20.233+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:20.291 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:20.305+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:20.353 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.365939815000004, 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.157789648000005, 2020-11-16T22:31:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.282078565000006, 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.411423600999996, 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.626706867000003, 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.077073792999997, 2020-11-16T22:31:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.076978353000001, 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.074444020999998, 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.724024227999998, 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.616245698999997, 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:31:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:31:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:31:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:31:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:31:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.354338337999998, 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.353604115000003, 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.291724876000004, 2020-11-16T22:31:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.021629390999998, 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.281118510999999, 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.822172052999999, 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 16.140101072, 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 16.139655199, 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.228458140000001, 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.4480003199999998, 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:31:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:31:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:31:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:31:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:20.766 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:20.767 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:20.781+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:20.767 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:20.868 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:20.882+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:20.903 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:20.903 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:20.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:27495 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 1705@2231224] snapc 1=[] ondisk+write+known_if_redirected+full_force e144)", 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:20.914817+0000", 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0030203389999999999, 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030475290000000002, 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:20.904 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914817+0000", 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914817+0000", 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4536999999999999e-05 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914841+0000", 2020-11-16T22:31:20.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2670000000000003e-06 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914847+0000", 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1168e-05 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914859+0000", 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2360000000000004e-06 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914864+0000", 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0403e-05 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914894+0000", 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3359999999999997e-05 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914948+0000", 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9745000000000002e-05 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.915037+0000", 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022071600000000001 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:45888 4.1 4:8c3af0c3:::1000000188a.00000000:head [read 1011712~28672 [1705@2231224]] snapc 0=[] ondisk+read+known_if_redirected e144)", 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:20.914910+0000", 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0029272220000000002, 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0030199609999999998, 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "delayed", 2020-11-16T22:31:20.912 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 45888 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914910+0000", 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:20.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914910+0000", 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9724e-05 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914929+0000", 2020-11-16T22:31:20.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8999999999999999e-06 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914933+0000", 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2250000000000003e-06 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:20.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914941+0000", 2020-11-16T22:31:20.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5750000000000001e-06 2020-11-16T22:31:20.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.914944+0000", 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0019760789999999999 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:20.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.916920+0000", 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103819 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:20.917024+0000", 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0021143220000000001 2020-11-16T22:31:20.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:31:20.920 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:31:20.922 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:20.968 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:20.983+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.068 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:21.083+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.069 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:21.070 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:21.070 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:21.070 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:21.070 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:21.080 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:21.169 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:21.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.270 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:21.284+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:21.283 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.976294822, 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:21.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:21.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:21.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:21.294 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 190.975791934, 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:21.295 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:21.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:21.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:21.303 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:21.304 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.047120702, 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:21.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:21.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:21.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:21.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.314 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.32983423100001, 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:21.315 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:21.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:21.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:21.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.325 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08468060199999, 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:21.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:21.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:21.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:21.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.335 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.084605482, 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:21.336 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:21.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:21.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:21.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.346 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08452654800001, 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:21.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:21.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:21.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:21.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.356 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08444912600001, 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:21.357 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.358 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.358 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:21.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:21.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:21.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:21.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:21.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.367 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08436896399999, 2020-11-16T22:31:21.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:21.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:21.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:21.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:21.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.378 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08428715100001, 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:21.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:21.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:21.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.388 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08419376500001, 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:21.389 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:21.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:21.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.08407336900001, 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.400 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:21.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:21.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:21.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.409 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.083973302, 2020-11-16T22:31:21.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:21.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:21.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:21.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:21.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:21.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 140.68370031800001, 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.421 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:21.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:21.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:21.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:21.430 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:21.431 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.83772738499999, 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:21.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:21.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:21.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:21.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:21.440 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.386925243999997, 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:21.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:21.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:21.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:21.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:21.449 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:21.450 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 82.088548184999993, 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:21.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 21.211600171000001, 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:21.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:21.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 16.159002184999999, 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:21.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:21.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:21.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.478 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 16.157958859000001, 2020-11-16T22:31:21.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:21.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:21.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:21.488 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:21.490 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:21.490 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:21.504+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.567 INFO:tasks.workunit.client.0.smithi138.stdout:30000 punch from 0xa93a7 to 0xacc3d, (0x3896 bytes) 2020-11-16T22:31:21.589 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:21.604+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.623 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:27648 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1729@890639] snapc 1=[] ondisk+write+known_if_redirected+full_force e145)", 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.634564+0000", 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0034568329999999999, 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034753549999999999, 2020-11-16T22:31:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634564+0000", 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634564+0000", 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2160000000000002e-05 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634616+0000", 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0361e-05 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634627+0000", 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6946e-05 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634644+0000", 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.571e-06 2020-11-16T22:31:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634651+0000", 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3139999999999997e-05 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634695+0000", 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9091000000000001e-05 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634754+0000", 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010481599999999999 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.634858+0000", 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019160410000000001 2020-11-16T22:31:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.636774+0000", 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022101260000000002 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:46099 4.0 4:0caefcac:::1000000188a.00000001:head [read 2809856~0 [1729@4194304]] snapc 0=[] ondisk+read+known_if_redirected e145)", 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.637599+0000", 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00042245800000000001, 2020-11-16T22:31:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00049401000000000002, 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 46099 2020-11-16T22:31:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637599+0000", 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637599+0000", 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7911000000000003e-05 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637647+0000", 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8310000000000001e-06 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637654+0000", 2020-11-16T22:31:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3668e-05 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637668+0000", 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2590000000000003e-06 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637675+0000", 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021328699999999999 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637889+0000", 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0704000000000004e-05 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637979+0000", 2020-11-16T22:31:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011335 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638093+0000", 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00049401000000000002 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:46100 4.5 4:a611fa68:::1000000188a.00000002:head [write 1901316~23814 [1729@890639] in=23814b] snapc 1=[] ondisk+write+known_if_redirected e145)", 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.637736+0000", 2020-11-16T22:31:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00028514300000000002, 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000449939, 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 46100 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637736+0000", 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637736+0000", 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7568e-05 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637814+0000", 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4240000000000004e-06 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637821+0000", 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4564e-05 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637836+0000", 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8939999999999999e-06 2020-11-16T22:31:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637844+0000", 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8217000000000003e-05 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637893+0000", 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5196000000000004e-05 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637958+0000", 2020-11-16T22:31:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012916100000000001 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638087+0000", 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035102400000000002 2020-11-16T22:31:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:46101 4.0 4:0caefcac:::1000000188a.00000001:head [write 2621440~61440 [1729@4194304] in=61440b] snapc 1=[] ondisk+write+known_if_redirected e145)", 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.637880+0000", 2020-11-16T22:31:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000141304, 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036742399999999998, 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 46101 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637880+0000", 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637880+0000", 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6235e-05 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637926+0000", 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4120000000000003e-06 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637931+0000", 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4830000000000002e-06 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637938+0000", 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2739999999999999e-06 2020-11-16T22:31:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.637941+0000", 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017892299999999999 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638120+0000", 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010319600000000001 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638223+0000", 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034352299999999998 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(mds.0.3:27648 4.5 e145/117)", 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.638075+0000", 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9999466, 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025481400000000002, 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638075+0000", 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638075+0000", 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638122+0000", 2020-11-16T22:31:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1799999999999996e-06 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638130+0000", 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3552e-05 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638144+0000", 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5559999999999998e-06 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638149+0000", 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4597000000000004e-05 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:46102 4.0 4.353f7530 (undecoded) ondisk+read+known_if_redirected e145)", 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:21.638261+0000", 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9997606, 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011022, 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:31:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 46102 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638261+0000", 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638261+0000", 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9538e-05 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638300+0000", 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3719999999999999e-06 2020-11-16T22:31:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638308+0000", 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.307e-05 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638321+0000", 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3859999999999998e-06 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:21.638326+0000", 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5365999999999997e-05 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 6 2020-11-16T22:31:21.673 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:21.673 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:21.717 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:21.731+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:21.791 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:21.806+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:21.815 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:21.816 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:21.825 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:21.908 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:21.922+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:21.992 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.006+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.002 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.015761674000004, 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.807611507000004, 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:22.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:22.028 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.931900424000006, 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:22.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:22.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:22.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.061245459999995, 2020-11-16T22:31:22.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:22.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:22.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:22.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 52.276528726000002, 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:22.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:22.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:22.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.726895652000003, 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.055 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:22.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:22.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:22.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:22.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.065 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.726800212000001, 2020-11-16T22:31:22.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:22.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:22.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:22.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:22.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.724265879999997, 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:22.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:22.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:22.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:22.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:22.088 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.373846086999997, 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:22.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:22.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:22.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.266067558000003, 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:31:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:31:22.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:31:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.109 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.004160196999997, 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:22.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:22.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:22.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.003425974000002, 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:22.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:22.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:22.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.941546735000003, 2020-11-16T22:31:22.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:22.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:22.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:22.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:22.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.671451249999997, 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:22.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:22.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.930940369999998, 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:22.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:22.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:22.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:22.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:22.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.165 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.471993911999999, 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:22.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:22.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.789922931, 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:22.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:22.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:22.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:22.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.789477057999999, 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:22.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:22.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 16.878279999, 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:22.197 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:22.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:22.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.0978221789999996, 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:22.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:22.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:22.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:31:22.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:31:22.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:22.218 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:22.218 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:22.219 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:22.220 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.233+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.220 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:22.319 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.334+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.357 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:22.357 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:22.357 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:31:22.357 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:22.367 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:22.368 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:22.388 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:22.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:22.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:22.420 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:22.434+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:22.521 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.535+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.556 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:22.557 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:22.569 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:22.651 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.666+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.722 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:22.736+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:22.766 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:22.767 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:22.778 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:22.823 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:22.837+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:22.916 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:22.916 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:22.924 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:22.938+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:22.925 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:31:22.926 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:23.025 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:23.039+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:23.071 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:31:23.072 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:31:23.072 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.5669855825149885 2020-11-16T22:31:23.072 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.5669855825149885 2020-11-16T22:31:23.113 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.113 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.122 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:23.126 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:23.139+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:23.227 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:23.240+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:23.299 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.300 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.309 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:23.353 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:23.367+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:23.428 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:23.442+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:23.453 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.454 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.464 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:23.547 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:23.561+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:23.630 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:23.644+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:23.641 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.641 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.651 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:23.730 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:23.745+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:23.831 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:23.845+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:23.836 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.837 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.846 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:23.932 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:23.947+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 0.566986 (9125) 2020-11-16T22:31:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:31:23.986 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:23.987 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:23.996 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:24.038 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:24.052+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:24.134 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:24.148+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:24.161 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:24.161 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:24.170 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:24.235 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:24.248+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:24.294 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:24.294 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:24.303 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:24.336 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:24.350+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:24.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2020-11-16T22:31:24.392 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:24.428 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:24.428 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:24.437 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:24.437 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:24.451+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:24.535 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:24.537 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:24.551+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:24.555 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:24.555 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:24.555 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:24.556 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:24.556 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:24.557 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:24.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:24.562 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:24.562 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:24.563 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:24.563 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:24.563 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:24.572 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:24.638 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:24.652+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:24.698 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:24.699 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 199.39118637799999, 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:24.700 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:24.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:24.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:24.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:24.709 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 194.39068348999999, 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:24.710 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:24.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:24.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:24.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 188.46201225799999, 2020-11-16T22:31:24.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:24.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:24.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:24.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:24.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 187.74472578699999, 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.730 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:24.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:24.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:24.739 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:24.740 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.49957215800001, 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.49949703799999, 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.751 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:24.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:24.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:24.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:24.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.761 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.499418104, 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:24.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:24.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:24.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:24.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.499340682, 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.772 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:24.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:24.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:24.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:24.781 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:24.782 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.49926052000001, 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:24.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:24.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:24.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.499178707, 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.793 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:24.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:24.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:24.802 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:24.803 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.499085321, 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:24.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:24.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:24.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:24.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:24.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.498964925, 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:24.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:25.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:25.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.025 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.49886485799999, 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:25.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:25.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:25.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:25.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.035 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 144.09859187399999, 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:25.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:25.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.25261894099999, 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:25.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:25.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.801816799999997, 2020-11-16T22:31:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:25.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:25.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:25.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:25.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:25.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:25.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:25.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.876 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.503439740999994, 2020-11-16T22:31:25.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:26.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:26.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:26.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:26.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:26.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 24.626491727000001, 2020-11-16T22:31:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.573893740999999, 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:26.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.924 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.572850415000001, 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:26.925 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:26.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:26.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:27.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:27.134 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:27.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:27.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:27.135 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:27.135 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:27.136 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:31:27.137 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:27.138 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:27.150+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:27.139 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:27.237 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.250+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.282 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.4 149/147 [PushOp(4:3f58eabc:::10000001451.00000000:head, version: 88'554, data_included: [0~1840], data_size: 1840, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3f58eabc:::10000001451.00000000:head@88'554, size: 1840, copy_subset: [0~1840], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1840, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:27.297357+0000", 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00026017999999999999, 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028754900000000001, 2020-11-16T22:31:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297357+0000", 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297357+0000", 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2076000000000001e-05 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297409+0000", 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297423+0000", 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2337999999999999e-05 2020-11-16T22:31:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297445+0000", 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0624e-05 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297456+0000", 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8925000000000003e-05 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297505+0000", 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7329999999999997e-06 2020-11-16T22:31:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:27.297511+0000", 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015417499999999999 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:31:27.290 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:27.293 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:27.495 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:27.495 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:27.495 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:27.495 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:27.496 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:27.496 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:27.496 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:27.496 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:27.496 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.508+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.595 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.608+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.669 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:27.669 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:27.679 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:27.696 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.710+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.797 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.810+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.820 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:27.820 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:27.831 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:27.897 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:27.911+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:27.985 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:27.985 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:27.996 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:28.005 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:31:28.037 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.051+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.099 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:28.114+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:28.191 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:28.191 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:28.200 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:28.214+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:28.202 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:28.301 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:28.315+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:28.362 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:28.362 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:28.374 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:28.417 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:28.432+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:28.503 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.516+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.534 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:28.535 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:28.546 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:28.603 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.617+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.704 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.717+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.739 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:28.739 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:28.750 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:28.804 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.818+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.905 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:28.918+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:28.911 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:28.911 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:28.923 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:28.995 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:31:28.995 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:31:28.995 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:31:29.005 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:29.019+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:29.091 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:29.091 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:29.104 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:29.106 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:29.120+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:29.206 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:29.220+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:29.259 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:29.259 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:29.271 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:29.303 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:31:29.305 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:31:29.312 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:29.326+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:29.316 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2020-11-16T22:31:28.928049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104395556,"num_objects":3599,"num_object_clones":0,"num_object_copies":7198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3599,"num_whiteouts":0,"num_read":27061,"num_read_kb":911651,"num_write":54155,"num_write_kb":580387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4881,"num_bytes_recovered":122867808,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46317,"ondisk_log_size":46317,"up":40,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":483036,"kb_used_data":158852,"kb_used_omap":26,"kb_used_meta":324133,"kb_avail":282632484,"statfs":{"total":289910292480,"available":289415663616,"internally_reserved":0,"allocated":162664448,"data_stored":203766928,"data_compressed":18648281,"data_compressed_allocated":64774144,"data_compressed_original":129535694,"omap_allocated":27325,"internal_metadata":331912515},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2485253,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":267,"num_read_kb":11539,"num_write":274,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":8733157,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.930226"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"2532","reported_epoch":"145","state":"active+clean","last_fresh":"2020-11-16T22:31:20.927860+0000","last_change":"2020-11-16T22:30:35.988507+0000","last_active":"2020-11-16T22:31:20.927860+0000","last_peered":"2020-11-16T22:31:20.927860+0000","last_clean":"2020-11-16T22:31:20.927860+0000","last_became_active":"2020-11-16T22:30:35.984396+0000","last_became_peered":"2020-11-16T22:30:35.984396+0000","last_unstale":"2020-11-16T22:31:20.927860+0000","last_undegraded":"2020-11-16T22:31:20.927860+0000","last_fullsized":"2020-11-16T22:31:20.927860+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"145","state":"active+undersized","last_fresh":"2020-11-16T22:31:20.928238+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:31:20.928238+0000","last_peered":"2020-11-16T22:31:20.928238+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:31:20.928238+0000","last_undegraded":"2020-11-16T22:31:20.928238+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"144'3202","reported_seq":"3328","reported_epoch":"148","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:26.003067+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:31:26.003067+0000","last_peered":"2020-11-16T22:31:26.003067+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:31:26.003067+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":3202,"ondisk_log_size":3202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583072,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3199,"num_write_kb":12760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"2364","reported_epoch":"143","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:06.122015+0000","last_change":"2020-11-16T22:31:04.115486+0000","last_active":"2020-11-16T22:31:06.122015+0000","last_peered":"2020-11-16T22:31:06.122015+0000","last_clean":"2020-11-16T22:29:54.995271+0000","last_became_active":"2020-11-16T22:31:04.115486+0000","last_became_peered":"2020-11-16T22:31:04.115486+0000","last_unstale":"2020-11-16T22:31:06.122015+0000","last_undegraded":"2020-11-16T22:31:04.111066+0000","last_fullsized":"2020-11-16T22:31:04.110910+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":1876427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":442}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"78","reported_epoch":"148","state":"active+undersized","last_fresh":"2020-11-16T22:31:26.002709+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:31:26.002709+0000","last_peered":"2020-11-16T22:31:26.002709+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:31:26.002709+0000","last_undegraded":"2020-11-16T22:31:26.002709+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4487","reported_epoch":"148","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:26.003120+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:31:26.003120+0000","last_peered":"2020-11-16T22:31:26.003120+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:31:26.003120+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1148","reported_epoch":"148","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:26.003167+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:31:26.003167+0000","last_peered":"2020-11-16T22:31:26.003167+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:31:26.003167+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"145","state":"active+undersized","last_fresh":"2020-11-16T22:31:20.927784+0000","last_change":"2020-11-16T22:31:04.115597+0000","last_active":"2020-11-16T22:31:20.927784+0000","last_peered":"2020-11-16T22:31:20.927784+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115597+0000","last_became_peered":"2020-11-16T22:31:04.115597+0000","last_unstale":"2020-11-16T22:31:20.927784+0000","last_undegraded":"2020-11-16T22:31:20.927784+0000","last_fullsized":"2020-11-16T22:31:04.110788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"140'8651","reported_seq":"8814","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002761+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:31:26.002761+0000","last_peered":"2020-11-16T22:31:26.002761+0000","last_clean":"2020-11-16T22:31:26.002761+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:31:26.002761+0000","last_undegraded":"2020-11-16T22:31:26.002761+0000","last_fullsized":"2020-11-16T22:31:26.002761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8651,"ondisk_log_size":8651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":17478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"148'3143","reported_seq":"4581","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:27.002639+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:31:27.002639+0000","last_peered":"2020-11-16T22:31:27.002639+0000","last_clean":"2020-11-16T22:31:27.002639+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:31:27.002639+0000","last_undegraded":"2020-11-16T22:31:27.002639+0000","last_fullsized":"2020-11-16T22:31:27.002639+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3143,"ondisk_log_size":3143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2989552,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":1305,"num_read_kb":43816,"num_write":3414,"num_write_kb":38029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"91","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002748+0000","last_change":"2020-11-16T22:31:25.006580+0000","last_active":"2020-11-16T22:31:26.002748+0000","last_peered":"2020-11-16T22:31:26.002748+0000","last_clean":"2020-11-16T22:31:26.002748+0000","last_became_active":"2020-11-16T22:31:25.006225+0000","last_became_peered":"2020-11-16T22:31:25.006225+0000","last_unstale":"2020-11-16T22:31:26.002748+0000","last_undegraded":"2020-11-16T22:31:26.002748+0000","last_fullsized":"2020-11-16T22:31:26.002748+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"77","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002517+0000","last_change":"2020-11-16T22:30:09.100967+0000","last_active":"2020-11-16T22:31:26.002517+0000","last_peered":"2020-11-16T22:31:26.002517+0000","last_clean":"2020-11-16T22:31:26.002517+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:31:26.002517+0000","last_undegraded":"2020-11-16T22:31:26.002517+0000","last_fullsized":"2020-11-16T22:31:26.002517+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"145'2933","reported_seq":"3011","reported_epoch":"146","state":"peering","last_fresh":"2020-11-16T22:31:23.994832+0000","last_change":"2020-11-16T22:31:23.995042+0000","last_active":"2020-11-16T22:31:23.994832+0000","last_peered":"2020-11-16T22:31:23.991736+0000","last_clean":"2020-11-16T22:29:54.994780+0000","last_became_active":"2020-11-16T22:31:04.115480+0000","last_became_peered":"2020-11-16T22:31:04.115480+0000","last_unstale":"2020-11-16T22:31:23.994832+0000","last_undegraded":"2020-11-16T22:31:23.994832+0000","last_fullsized":"2020-11-16T22:31:23.994832+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2933,"ondisk_log_size":2933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9854400,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2930,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1036","reported_epoch":"143","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:06.122254+0000","last_change":"2020-11-16T22:31:04.116677+0000","last_active":"2020-11-16T22:31:06.122254+0000","last_peered":"2020-11-16T22:31:06.122254+0000","last_clean":"2020-11-16T22:29:54.995625+0000","last_became_active":"2020-11-16T22:31:04.116677+0000","last_became_peered":"2020-11-16T22:31:04.116677+0000","last_unstale":"2020-11-16T22:31:06.122254+0000","last_undegraded":"2020-11-16T22:31:04.111161+0000","last_fullsized":"2020-11-16T22:31:04.110789+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":436}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"73","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002559+0000","last_change":"2020-11-16T22:30:09.102224+0000","last_active":"2020-11-16T22:31:26.002559+0000","last_peered":"2020-11-16T22:31:26.002559+0000","last_clean":"2020-11-16T22:31:26.002559+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:31:26.002559+0000","last_undegraded":"2020-11-16T22:31:26.002559+0000","last_fullsized":"2020-11-16T22:31:26.002559+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"189","reported_epoch":"148","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:26.002700+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:31:26.002700+0000","last_peered":"2020-11-16T22:31:26.002700+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:31:26.002700+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"26728","reported_epoch":"145","state":"active+clean","last_fresh":"2020-11-16T22:31:20.927889+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:31:20.927889+0000","last_peered":"2020-11-16T22:31:20.927889+0000","last_clean":"2020-11-16T22:31:20.927889+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:31:20.927889+0000","last_undegraded":"2020-11-16T22:31:20.927889+0000","last_fullsized":"2020-11-16T22:31:20.927889+0000","mapping_epoch":137,"log_start":"93'1500","ondisk_log_start":"93'1500","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10054,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"11","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002790+0000","last_change":"2020-11-16T22:31:25.006890+0000","last_active":"2020-11-16T22:31:26.002790+0000","last_peered":"2020-11-16T22:31:26.002790+0000","last_clean":"2020-11-16T22:31:26.002790+0000","last_became_active":"2020-11-16T22:31:25.006438+0000","last_became_peered":"2020-11-16T22:31:25.006438+0000","last_unstale":"2020-11-16T22:31:26.002790+0000","last_undegraded":"2020-11-16T22:31:26.002790+0000","last_fullsized":"2020-11-16T22:31:26.002790+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"144'71","reported_seq":"23","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.002746+0000","last_change":"2020-11-16T22:31:25.063953+0000","last_active":"2020-11-16T22:31:26.002746+0000","last_peered":"2020-11-16T22:31:26.002746+0000","last_clean":"2020-11-16T22:31:26.002746+0000","last_became_active":"2020-11-16T22:31:25.006560+0000","last_became_peered":"2020-11-16T22:31:25.006560+0000","last_unstale":"2020-11-16T22:31:26.002746+0000","last_undegraded":"2020-11-16T22:31:26.002746+0000","last_fullsized":"2020-11-16T22:31:26.002746+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":71,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"149'4969","reported_seq":"11942","reported_epoch":"149","state":"active+recovering+undersized+degraded+remapped","last_fresh":"2020-11-16T22:31:27.119928+0000","last_change":"2020-11-16T22:31:26.119811+0000","last_active":"2020-11-16T22:31:27.119928+0000","last_peered":"2020-11-16T22:31:27.119928+0000","last_clean":"2020-11-16T22:31:02.517481+0000","last_became_active":"2020-11-16T22:31:26.106409+0000","last_became_peered":"2020-11-16T22:31:26.106409+0000","last_unstale":"2020-11-16T22:31:27.119928+0000","last_undegraded":"2020-11-16T22:31:26.016448+0000","last_fullsized":"2020-11-16T22:31:26.002702+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4969,"ondisk_log_size":4969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6483990,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":5210,"num_read_kb":227019,"num_write":5525,"num_write_kb":78584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":3228085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":164}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"144'9","reported_seq":"61","reported_epoch":"145","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:20.928054+0000","last_change":"2020-11-16T22:31:04.115488+0000","last_active":"2020-11-16T22:31:20.928054+0000","last_peered":"2020-11-16T22:31:20.928054+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:31:20.928054+0000","last_undegraded":"2020-11-16T22:31:04.111159+0000","last_fullsized":"2020-11-16T22:31:04.111040+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"51","reported_epoch":"146","state":"peering","last_fresh":"2020-11-16T22:31:23.994235+0000","last_change":"2020-11-16T22:31:23.994432+0000","last_active":"2020-11-16T22:31:23.994235+0000","last_peered":"2020-11-16T22:31:20.927975+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113279+0000","last_became_peered":"2020-11-16T22:31:04.113279+0000","last_unstale":"2020-11-16T22:31:23.994235+0000","last_undegraded":"2020-11-16T22:31:23.994235+0000","last_fullsized":"2020-11-16T22:31:23.994235+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"146'4941","reported_seq":"11686","reported_epoch":"146","state":"active+clean","last_fresh":"2020-11-16T22:31:23.999300+0000","last_change":"2020-11-16T22:30:35.985757+0000","last_active":"2020-11-16T22:31:23.999300+0000","last_peered":"2020-11-16T22:31:23.999300+0000","last_clean":"2020-11-16T22:31:23.999300+0000","last_became_active":"2020-11-16T22:30:35.980986+0000","last_became_peered":"2020-11-16T22:30:35.980986+0000","last_unstale":"2020-11-16T22:31:23.999300+0000","last_undegraded":"2020-11-16T22:31:23.999300+0000","last_fullsized":"2020-11-16T22:31:23.999300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4941,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6243682,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":6410,"num_read_kb":291675,"num_write":5772,"num_write_kb":89406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"73","reported_epoch":"148","state":"active+clean","last_fresh":"2020-11-16T22:31:26.003320+0000","last_change":"2020-11-16T22:30:09.100836+0000","last_active":"2020-11-16T22:31:26.003320+0000","last_peered":"2020-11-16T22:31:26.003320+0000","last_clean":"2020-11-16T22:31:26.003320+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:31:26.003320+0000","last_undegraded":"2020-11-16T22:31:26.003320+0000","last_fullsized":"2020-11-16T22:31:26.003320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"144'107","reported_seq":"106","reported_epoch":"149","state":"peering","last_fresh":"2020-11-16T22:31:27.011931+0000","last_change":"2020-11-16T22:31:27.012165+0000","last_active":"2020-11-16T22:31:27.011931+0000","last_peered":"2020-11-16T22:31:26.096733+0000","last_clean":"2020-11-16T22:30:53.082925+0000","last_became_active":"2020-11-16T22:31:26.021711+0000","last_became_peered":"2020-11-16T22:31:26.021711+0000","last_unstale":"2020-11-16T22:31:27.011931+0000","last_undegraded":"2020-11-16T22:31:27.011931+0000","last_fullsized":"2020-11-16T22:31:27.011931+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":107,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":35601732,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":27035,"num_read_kb":911420,"num_write":34796,"num_write_kb":510313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4827,"num_bytes_recovered":26249540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68390912,"data_stored":56141160,"data_compressed":4854963,"data_compressed_allocated":9379840,"data_compressed_original":18747086,"omap_allocated":0,"internal_metadata":0},"log_size":26953,"ondisk_log_size":26953,"up":13,"acting":12,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":68793824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":19355,"num_write_kb":70054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":96618268,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90738688,"data_stored":146120804,"data_compressed":13800365,"data_compressed_allocated":55451648,"data_compressed_original":110903296,"omap_allocated":10534,"internal_metadata":0},"log_size":19360,"ondisk_log_size":19360,"up":12,"acting":12,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":13,"acting":13,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467987,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":151084,"kb_used_data":67300,"kb_used_omap":26,"kb_used_meta":83749,"kb_avail":94220756,"statfs":{"total":96636764160,"available":96482054144,"internally_reserved":0,"allocated":68915200,"data_stored":89735148,"data_compressed":8435115,"data_compressed_allocated":30060544,"data_compressed_original":60121088,"omap_allocated":27325,"internal_metadata":85759299},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109904,"kb_used_data":35336,"kb_used_omap":0,"kb_used_meta":74560,"kb_avail":94261936,"statfs":{"total":96636764160,"available":96524222464,"internally_reserved":0,"allocated":36184064,"data_stored":55423993,"data_compressed":6804590,"data_compressed_allocated":25083904,"data_compressed_original":50167808,"omap_allocated":0,"internal_metadata":76349440},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:30:20 2020","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":1.028},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738469,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":222048,"kb_used_data":56216,"kb_used_omap":0,"kb_used_meta":165824,"kb_avail":94149792,"statfs":{"total":96636764160,"available":96409387008,"internally_reserved":0,"allocated":57565184,"data_stored":58607787,"data_compressed":3408576,"data_compressed_allocated":9629696,"data_compressed_original":19246798,"omap_allocated":0,"internal_metadata":169803776},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:30:58 2020","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55000000000000004}]}]}]}} 2020-11-16T22:31:29.928 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:29.929 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:29.930 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:29.930 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:29.930 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:29.942+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:30.017 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:31:30.029 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:30.043+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:30.071 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:30.071 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:30.082 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:30.105 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:30.105 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:30.106 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:30.106 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:30.108 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:30.108 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:30.165 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:30.180+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:30.230 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:30.243+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:30.286 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:30.286 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:30.286 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:30.286 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:30.286 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:30.298 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:30.339 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:30.353+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:30.431 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:30.444+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:30.505 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:30.505 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:30.506 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.518473180000001, 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:30.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:30.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:30.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:30.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.310323013000001, 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:30.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:30.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:30.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:30.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:30.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:31.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:31.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:31.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:31.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:31.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:31.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:31.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:31.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:31.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:31.340 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.434611930000003, 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:31.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:31.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:31.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:31.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:31.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.754 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:31.754 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:31.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.563956966000006, 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:31.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:31.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:31.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:31.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:31.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:31.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:31.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.965 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.779240231999999, 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:32.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:32.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:32.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:32.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:32.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:32.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:32.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:32.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:32.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:32.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:32.382 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:32.383 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.229607158, 2020-11-16T22:31:32.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:32.383 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:32.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:32.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:32.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:32.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:32.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:32.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:32.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:32.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:32.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:32.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:32.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:32.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:32.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:32.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:32.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:32.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:32.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.002 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:33.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.229511717999998, 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:33.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:33.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:33.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:33.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:33.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:33.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:33.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:33.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:33.420 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.226977386000002, 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:33.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:33.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:33.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:33.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:33.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:33.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:33.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:33.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.876557593000001, 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:34.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:34.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:34.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:34.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:34.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:34.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:34.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:34.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:34.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:34.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:34.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:34.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:34.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18651 3.0 3:1e8917d7:::200.0000000e:head [write 1909287~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.768779064, 2020-11-16T22:31:34.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219, 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:34.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:34.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:34.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749762+0000", 2020-11-16T22:31:34.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2935999999999998e-05 2020-11-16T22:31:34.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749795+0000", 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0799999999999996e-06 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749800+0000", 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1720000000000007e-06 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749808+0000", 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-06 2020-11-16T22:31:34.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749812+0000", 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9227000000000001e-05 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749861+0000", 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0494000000000001e-05 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749912+0000", 2020-11-16T22:31:34.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4862000000000002e-05 2020-11-16T22:31:34.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:34.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.749997+0000", 2020-11-16T22:31:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015763489999999999 2020-11-16T22:31:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.751573+0000", 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001865748 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753439+0000", 2020-11-16T22:31:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9467e-05 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.753478+0000", 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010767993999999999 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.764246+0000", 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014484219 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.506871703000002, 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:34.874 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:34.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.874 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:34.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:34.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:34.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:34.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:34.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:34.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:34.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:34.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:35.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:35.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:35.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:35.086 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:35.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:35.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:35.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.087 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:35.290 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:35.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.50613748, 2020-11-16T22:31:35.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:35.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:35.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:35.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:35.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:35.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:35.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:35.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:35.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:35.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:35.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.444258241, 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:35.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:35.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:35.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:35.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:35.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:36.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:36.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:36.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:36.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.174162756000001, 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:36.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:36.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:36.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:36.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:36.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:36.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:36.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:36.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:36.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:36.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:36.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:36.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:36.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:36.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:36.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:36.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:36.540 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.433651875999999, 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:36.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.974705417999999, 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:37.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:37.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:37.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:37.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:37.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:37.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:37.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:37.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:37.374 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:37.375 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.292634437, 2020-11-16T22:31:37.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:37.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:37.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:37.578 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:37.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:37.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:37.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:37.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:37.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:37.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:37.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:37.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:37.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:37.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:37.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:37.790 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:37.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.791 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:37.994 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:37.994 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.292188564, 2020-11-16T22:31:37.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:37.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:37.994 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:37.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:37.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:37.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:37.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:37.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:37.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:37.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.380991505000001, 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:38.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:38.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:38.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:38.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:38.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:38.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:38.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:38.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:38.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:38.621 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.600533685, 2020-11-16T22:31:38.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:31:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:38.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:38.826 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:38.826 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:31:38.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:38.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:31:38.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:38.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:31:38.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:31:38.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:38.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:38.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:39.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:31:39.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:31:39.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:31:39.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:39.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:39.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:39.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:39.038 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:39.038 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:39.039 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:39.039 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:31:39.040 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:31:39.040 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:31:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:31:39.042 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:39.053+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:39.043 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:39.248 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:39.261+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:39.248 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:39.252 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:39.255 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:39.347 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:39.362+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:39.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~12s 2020-11-16T22:31:39.386 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:39.386 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:31:39.391 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 155, 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:31:33.025193+0000", 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:31:03.095061+0000", 2020-11-16T22:31:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5806080, 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "nodeep-scrub", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "noscrub", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:31:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:31:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:31:39.598 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:31:39.598 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:31:39.598 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:31:39.598 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "luminous", 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:31:39.599 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:31:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:31:39.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:31:39.602 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:31:39.603 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:31:39.806 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:31:39.806 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:31:39.806 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:31:39.807 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:31:39.808 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:31:39.809 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:31:39.809 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:31:39.809 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:31:39.809 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:39.810 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:31:39.811 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:31:39.812 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:31:39.812 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:31:39.812 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:31:39.813 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:31:39.813 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:31:40.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.014 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:31:40.014 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:31:40.015 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:40.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:40.015 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:31:40.015 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:31:40.016 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:31:40.016 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:31:40.016 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:31:40.016 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:31:40.016 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:31:40.017 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:31:40.017 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:31:40.017 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:31:40.017 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:31:40.017 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:31:40.018 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:31:40.018 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:31:40.018 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:31:40.018 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:31:40.018 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:31:40.019 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:31:40.019 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:31:40.019 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:31:40.019 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:31:40.019 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:31:40.020 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:31:40.020 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:31:40.020 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:31:40.020 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:31:40.020 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:31:40.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.021 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:31:40.021 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:31:40.021 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:31:40.021 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:31:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:31:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:31:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:31:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:31:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:31:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:31:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:31:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:31:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:40.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:40.430 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:31:40.430 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:31:40.430 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:31:40.431 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:31:40.432 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:31:40.433 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:31:40.434 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:31:40.435 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:31:40.638 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:31:40.638 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:31:40.639 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:31:40.639 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:31:40.639 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:31:40.639 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:31:40.640 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:31:40.640 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:31:40.640 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:31:40.640 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:31:40.640 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:31:40.641 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:31:40.641 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:31:40.641 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:31:40.641 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:31:40.641 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:31:40.642 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:31:40.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:31:40.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.642 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:31:40.642 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:31:40.643 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:31:40.643 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:31:40.643 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:31:40.643 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:31:40.644 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:31:40.644 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:31:40.644 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:31:40.644 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:31:40.644 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:31:40.645 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:31:40.645 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:31:40.645 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:31:40.645 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:31:40.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.646 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:31:40.646 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:31:40.646 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:31:40.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:40.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:40.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:31:40.851 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:31:40.852 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:31:40.853 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:31:40.854 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:31:40.855 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:31:40.856 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:31:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:31:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 0, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 0.5669708251953125, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:31:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 119, 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 141, 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:31:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:31:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.474 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.474 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.476 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.478 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: "exists" 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:31:41.480 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0.7154998779296875, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 153, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:31:41.481 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.482 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:31:41.483 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:31:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.486 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:31:41.489 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:31:41.491 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 63, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 69, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 108, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 154, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 70, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:31:41.492 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.493 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.495 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.497 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:31:41.499 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/2959", 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/2959", 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/2959", 2020-11-16T22:31:41.500 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/2959", 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:31:41.501 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:31:41.502 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 141 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:31:41.503 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.504 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:31:41.505 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 70 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [ 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.7", 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:31:41.506 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: 0 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [], 2020-11-16T22:31:41.507 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:31:41.508 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:31:41.509 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:31:41.510 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:31:41.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:31:41.511 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:31:41.511 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:31:41.512 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:41.514 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:41.515 INFO:tasks.thrashosds.thrasher:j is {'epoch': 155, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:31:33.025193+0000', 'last_up_change': '2020-11-16T22:31:03.095061+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 0, 'in': 1, 'weight': 0.5669708251953125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 119, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7154998779296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 63, 'last_clean_end': 69, 'up_from': 108, 'up_thru': 154, 'down_at': 70, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 2959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 2959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 2959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 2959}]}, 'public_addr': '172.21.15.122:6801/2959', 'cluster_addr': '172.21.15.122:6803/2959', 'heartbeat_back_addr': '172.21.15.122:6807/2959', 'heartbeat_front_addr': '172.21.15.122:6805/2959', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 70}], 'pg_upmap': [{'pgid': '4.7', 'osds': [2, 0]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:31:41.515 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:31:41.516 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:41.526+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:41.612 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:41.627+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:41.684 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:41.684 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:41.695 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:41.706 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:41.713 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:41.727+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:41.727 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:41.728 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:41.730 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:41.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:41.814 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:41.828+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:41.843 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:41.843 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.53572296900001, 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:41.844 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.845 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:41.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 211.53522008100001, 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.854 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:41.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:41.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.863 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 205.60654884900001, 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:41.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:41.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:41.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:41.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.873 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.88926237800001, 2020-11-16T22:31:41.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.876 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:41.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:41.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:41.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.644108749, 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64403362900001, 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.895 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:41.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:41.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64395469499999, 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:41.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:41.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:41.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.915 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64387727299999, 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.916 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:41.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:41.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:41.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.926 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.643797111, 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:41.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:41.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:41.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64371529799999, 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.937 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:41.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:41.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:41.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:41.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:41.947 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64362191199999, 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:41.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:41.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:41.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:41.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64350151599999, 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.958 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:41.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:41.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:41.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.967 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 202.64340144900001, 2020-11-16T22:31:41.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:41.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:41.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:41.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:41.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:41.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.978 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 161.24312846500001, 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:41.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:41.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:41.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:41.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.988 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 125.397155532, 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:41.989 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:41.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:41.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:41.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:41.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.997 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 102.946353391, 2020-11-16T22:31:41.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:41.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:42.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:42.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:42.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:42.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:42.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:42.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:42.008 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 102.647976332, 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:42.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:42.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:42.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:42.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:42.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:42.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:42.019 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:42.020 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.771028317999999, 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:42.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:42.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:42.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 36.718430331999997, 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:42.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:42.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:42.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:42.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.036 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 36.717387006000003, 2020-11-16T22:31:42.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:42.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:42.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:42.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.045 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:42.048 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:31:42.048 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2020-11-16T22:31:41.735551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101521977,"num_objects":3599,"num_object_clones":0,"num_object_copies":7198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3599,"num_whiteouts":0,"num_read":28775,"num_read_kb":992906,"num_write":56834,"num_write_kb":608332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6804,"num_bytes_recovered":139054382,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":48855,"ondisk_log_size":48855,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":531360,"kb_used_data":172936,"kb_used_omap":26,"kb_used_meta":358373,"kb_avail":282584160,"statfs":{"total":289910292480,"available":289366179840,"internally_reserved":0,"allocated":177086464,"data_stored":218492380,"data_compressed":19867851,"data_compressed_allocated":68362240,"data_compressed_original":136699848,"omap_allocated":27325,"internal_metadata":366974275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1028658,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1487,"num_read_kb":69768,"num_write":2326,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1572,"num_bytes_recovered":5320241,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.709630"},"pg_stats":[{"pgid":"4.2","version":"92'971","reported_seq":"2541","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029297+0000","last_change":"2020-11-16T22:30:35.988507+0000","last_active":"2020-11-16T22:31:32.029297+0000","last_peered":"2020-11-16T22:31:32.029297+0000","last_clean":"2020-11-16T22:31:32.029297+0000","last_became_active":"2020-11-16T22:30:35.984396+0000","last_became_peered":"2020-11-16T22:30:35.984396+0000","last_unstale":"2020-11-16T22:31:32.029297+0000","last_undegraded":"2020-11-16T22:31:32.029297+0000","last_fullsized":"2020-11-16T22:31:32.029297+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":971,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"21","reported_epoch":"154","state":"active+undersized","last_fresh":"2020-11-16T22:31:32.029424+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:31:32.029424+0000","last_peered":"2020-11-16T22:31:32.029424+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:31:32.029424+0000","last_undegraded":"2020-11-16T22:31:32.029424+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"144'3202","reported_seq":"3334","reported_epoch":"154","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:32.029755+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:31:32.029755+0000","last_peered":"2020-11-16T22:31:32.029755+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:31:32.029755+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":3202,"ondisk_log_size":3202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583072,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3199,"num_write_kb":12760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"92'936","reported_seq":"2382","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029288+0000","last_change":"2020-11-16T22:31:29.946505+0000","last_active":"2020-11-16T22:31:32.029288+0000","last_peered":"2020-11-16T22:31:32.029288+0000","last_clean":"2020-11-16T22:31:32.029288+0000","last_became_active":"2020-11-16T22:31:29.943620+0000","last_became_peered":"2020-11-16T22:31:29.943620+0000","last_unstale":"2020-11-16T22:31:32.029288+0000","last_undegraded":"2020-11-16T22:31:32.029288+0000","last_fullsized":"2020-11-16T22:31:32.029288+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":936,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1916343,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1163,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"84","reported_epoch":"154","state":"active+undersized","last_fresh":"2020-11-16T22:31:32.029168+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:31:32.029168+0000","last_peered":"2020-11-16T22:31:32.029168+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:31:32.029168+0000","last_undegraded":"2020-11-16T22:31:32.029168+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"107'4359","reported_seq":"4493","reported_epoch":"154","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:32.029435+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:31:32.029435+0000","last_peered":"2020-11-16T22:31:32.029435+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:31:32.029435+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4359,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"92'984","reported_seq":"1154","reported_epoch":"154","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:32.029404+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:31:32.029404+0000","last_peered":"2020-11-16T22:31:32.029404+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:31:32.029404+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":984,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1847049,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":2813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"21","reported_epoch":"154","state":"active+undersized","last_fresh":"2020-11-16T22:31:32.029281+0000","last_change":"2020-11-16T22:31:04.115597+0000","last_active":"2020-11-16T22:31:32.029281+0000","last_peered":"2020-11-16T22:31:32.029281+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115597+0000","last_became_peered":"2020-11-16T22:31:04.115597+0000","last_unstale":"2020-11-16T22:31:32.029281+0000","last_undegraded":"2020-11-16T22:31:32.029281+0000","last_fullsized":"2020-11-16T22:31:04.110788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"140'8651","reported_seq":"8820","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029072+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:31:32.029072+0000","last_peered":"2020-11-16T22:31:32.029072+0000","last_clean":"2020-11-16T22:31:32.029072+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:31:32.029072+0000","last_undegraded":"2020-11-16T22:31:32.029072+0000","last_fullsized":"2020-11-16T22:31:32.029072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":8651,"ondisk_log_size":8651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777216,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":17478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"155'3397","reported_seq":"4987","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:37.031559+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:31:37.031559+0000","last_peered":"2020-11-16T22:31:37.031559+0000","last_clean":"2020-11-16T22:31:37.031559+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:31:37.031559+0000","last_undegraded":"2020-11-16T22:31:37.031559+0000","last_fullsized":"2020-11-16T22:31:37.031559+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3397,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2924420,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":1451,"num_read_kb":48473,"num_write":3688,"num_write_kb":42575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"97","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029092+0000","last_change":"2020-11-16T22:31:25.006580+0000","last_active":"2020-11-16T22:31:32.029092+0000","last_peered":"2020-11-16T22:31:32.029092+0000","last_clean":"2020-11-16T22:31:32.029092+0000","last_became_active":"2020-11-16T22:31:25.006225+0000","last_became_peered":"2020-11-16T22:31:25.006225+0000","last_unstale":"2020-11-16T22:31:32.029092+0000","last_undegraded":"2020-11-16T22:31:32.029092+0000","last_fullsized":"2020-11-16T22:31:32.029092+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"83","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.028868+0000","last_change":"2020-11-16T22:30:09.100967+0000","last_active":"2020-11-16T22:31:32.028868+0000","last_peered":"2020-11-16T22:31:32.028868+0000","last_clean":"2020-11-16T22:31:32.028868+0000","last_became_active":"2020-11-16T22:29:58.923030+0000","last_became_peered":"2020-11-16T22:29:58.923030+0000","last_unstale":"2020-11-16T22:31:32.028868+0000","last_undegraded":"2020-11-16T22:31:32.028868+0000","last_fullsized":"2020-11-16T22:31:32.028868+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"155'4131","reported_seq":"4240","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:39.654219+0000","last_change":"2020-11-16T22:31:28.026885+0000","last_active":"2020-11-16T22:31:39.654219+0000","last_peered":"2020-11-16T22:31:39.654219+0000","last_clean":"2020-11-16T22:31:39.654219+0000","last_became_active":"2020-11-16T22:31:28.025691+0000","last_became_peered":"2020-11-16T22:31:28.025691+0000","last_unstale":"2020-11-16T22:31:39.654219+0000","last_undegraded":"2020-11-16T22:31:39.654219+0000","last_fullsized":"2020-11-16T22:31:39.654219+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4131,"ondisk_log_size":4131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12179239,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4128,"num_write_kb":12414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"92'955","reported_seq":"1051","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:33.036238+0000","last_change":"2020-11-16T22:31:33.036238+0000","last_active":"2020-11-16T22:31:33.036238+0000","last_peered":"2020-11-16T22:31:33.036238+0000","last_clean":"2020-11-16T22:31:33.036238+0000","last_became_active":"2020-11-16T22:31:33.033251+0000","last_became_peered":"2020-11-16T22:31:33.033251+0000","last_unstale":"2020-11-16T22:31:33.036238+0000","last_undegraded":"2020-11-16T22:31:33.036238+0000","last_fullsized":"2020-11-16T22:31:33.036238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":955,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":3955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"79","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.028893+0000","last_change":"2020-11-16T22:30:09.102224+0000","last_active":"2020-11-16T22:31:32.028893+0000","last_peered":"2020-11-16T22:31:32.028893+0000","last_clean":"2020-11-16T22:31:32.028893+0000","last_became_active":"2020-11-16T22:29:58.922737+0000","last_became_peered":"2020-11-16T22:29:58.922737+0000","last_unstale":"2020-11-16T22:31:32.028893+0000","last_undegraded":"2020-11-16T22:31:32.028893+0000","last_fullsized":"2020-11-16T22:31:32.028893+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"195","reported_epoch":"154","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:32.029078+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:31:32.029078+0000","last_peered":"2020-11-16T22:31:32.029078+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:31:32.029078+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"107'11554","reported_seq":"26737","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029361+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:31:32.029361+0000","last_peered":"2020-11-16T22:31:32.029361+0000","last_clean":"2020-11-16T22:31:32.029361+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:31:32.029361+0000","last_undegraded":"2020-11-16T22:31:32.029361+0000","last_fullsized":"2020-11-16T22:31:32.029361+0000","mapping_epoch":137,"log_start":"93'1500","ondisk_log_start":"93'1500","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10054,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":14110,"num_read_kb":348910,"num_write":15534,"num_write_kb":286884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"17","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029535+0000","last_change":"2020-11-16T22:31:25.006890+0000","last_active":"2020-11-16T22:31:32.029535+0000","last_peered":"2020-11-16T22:31:32.029535+0000","last_clean":"2020-11-16T22:31:32.029535+0000","last_became_active":"2020-11-16T22:31:25.006438+0000","last_became_peered":"2020-11-16T22:31:25.006438+0000","last_unstale":"2020-11-16T22:31:32.029535+0000","last_undegraded":"2020-11-16T22:31:32.029535+0000","last_fullsized":"2020-11-16T22:31:32.029535+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"154'72","reported_seq":"30","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.088585+0000","last_change":"2020-11-16T22:31:25.063953+0000","last_active":"2020-11-16T22:31:32.088585+0000","last_peered":"2020-11-16T22:31:32.088585+0000","last_clean":"2020-11-16T22:31:32.088585+0000","last_became_active":"2020-11-16T22:31:25.006560+0000","last_became_peered":"2020-11-16T22:31:25.006560+0000","last_unstale":"2020-11-16T22:31:32.088585+0000","last_undegraded":"2020-11-16T22:31:32.088585+0000","last_fullsized":"2020-11-16T22:31:32.088585+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194947,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70,"num_write_kb":4135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"155'5418","reported_seq":"14081","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:37.096880+0000","last_change":"2020-11-16T22:31:32.041223+0000","last_active":"2020-11-16T22:31:37.096880+0000","last_peered":"2020-11-16T22:31:37.096880+0000","last_clean":"2020-11-16T22:31:37.096880+0000","last_became_active":"2020-11-16T22:31:32.033347+0000","last_became_peered":"2020-11-16T22:31:32.033347+0000","last_unstale":"2020-11-16T22:31:37.096880+0000","last_undegraded":"2020-11-16T22:31:37.096880+0000","last_fullsized":"2020-11-16T22:31:37.096880+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":5418,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2919944,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":5777,"num_read_kb":250944,"num_write":6010,"num_write_kb":86611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"144'9","reported_seq":"70","reported_epoch":"154","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:31:32.029712+0000","last_change":"2020-11-16T22:31:04.115488+0000","last_active":"2020-11-16T22:31:32.029712+0000","last_peered":"2020-11-16T22:31:32.029712+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:31:32.029712+0000","last_undegraded":"2020-11-16T22:31:04.111159+0000","last_fullsized":"2020-11-16T22:31:04.111040+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"67","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029683+0000","last_change":"2020-11-16T22:31:25.006469+0000","last_active":"2020-11-16T22:31:32.029683+0000","last_peered":"2020-11-16T22:31:32.029683+0000","last_clean":"2020-11-16T22:31:32.029683+0000","last_became_active":"2020-11-16T22:31:25.006214+0000","last_became_peered":"2020-11-16T22:31:25.006214+0000","last_unstale":"2020-11-16T22:31:32.029683+0000","last_undegraded":"2020-11-16T22:31:32.029683+0000","last_fullsized":"2020-11-16T22:31:32.029683+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"155'5576","reported_seq":"13331","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:39.664543+0000","last_change":"2020-11-16T22:30:35.985757+0000","last_active":"2020-11-16T22:31:39.664543+0000","last_peered":"2020-11-16T22:31:39.664543+0000","last_clean":"2020-11-16T22:31:39.664543+0000","last_became_active":"2020-11-16T22:30:35.980986+0000","last_became_peered":"2020-11-16T22:30:35.980986+0000","last_unstale":"2020-11-16T22:31:39.664543+0000","last_undegraded":"2020-11-16T22:31:39.664543+0000","last_fullsized":"2020-11-16T22:31:39.664543+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":5576,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4674442,"num_objects":456,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":456,"num_whiteouts":0,"num_read":7411,"num_read_kb":344348,"num_write":6492,"num_write_kb":102356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"79","reported_epoch":"154","state":"active+clean","last_fresh":"2020-11-16T22:31:32.029401+0000","last_change":"2020-11-16T22:30:09.100836+0000","last_active":"2020-11-16T22:31:32.029401+0000","last_peered":"2020-11-16T22:31:32.029401+0000","last_clean":"2020-11-16T22:31:32.029401+0000","last_became_active":"2020-11-16T22:29:58.922577+0000","last_became_peered":"2020-11-16T22:29:58.922577+0000","last_unstale":"2020-11-16T22:31:32.029401+0000","last_undegraded":"2020-11-16T22:31:32.029401+0000","last_fullsized":"2020-11-16T22:31:32.029401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"155'108","reported_seq":"120","reported_epoch":"155","state":"active+clean","last_fresh":"2020-11-16T22:31:33.029261+0000","last_change":"2020-11-16T22:31:28.025847+0000","last_active":"2020-11-16T22:31:33.029261+0000","last_peered":"2020-11-16T22:31:33.029261+0000","last_clean":"2020-11-16T22:31:33.029261+0000","last_became_active":"2020-11-16T22:31:28.025465+0000","last_became_peered":"2020-11-16T22:31:28.025465+0000","last_unstale":"2020-11-16T22:31:33.029261+0000","last_undegraded":"2020-11-16T22:31:33.029261+0000","last_fullsized":"2020-11-16T22:31:33.029261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":108,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":30403314,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":28749,"num_read_kb":992675,"num_write":36275,"num_write_kb":535836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76349440,"data_stored":62535549,"data_compressed":5618520,"data_compressed_allocated":10821632,"data_compressed_original":21616812,"omap_allocated":0,"internal_metadata":0},"log_size":28291,"ondisk_log_size":28291,"up":15,"acting":15,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":71118663,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":20555,"num_write_kb":72476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":106481772,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96849920,"data_stored":154325149,"data_compressed":14251271,"data_compressed_allocated":57548800,"data_compressed_original":115097600,"omap_allocated":10534,"internal_metadata":0},"log_size":20560,"ondisk_log_size":20560,"up":12,"acting":12,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":13,"acting":13,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856467989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175476,"kb_used_data":73772,"kb_used_omap":26,"kb_used_meta":101669,"kb_avail":94196364,"statfs":{"total":96636764160,"available":96457076736,"internally_reserved":0,"allocated":75542528,"data_stored":95031749,"data_compressed":8735839,"data_compressed_allocated":30703616,"data_compressed_original":61400283,"omap_allocated":27325,"internal_metadata":104109379},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.34100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109904,"kb_used_data":35336,"kb_used_omap":0,"kb_used_meta":74560,"kb_avail":94261936,"statfs":{"total":96636764160,"available":96524222464,"internally_reserved":0,"allocated":36184064,"data_stored":55423993,"data_compressed":6804590,"data_compressed_allocated":25083904,"data_compressed_original":50167808,"omap_allocated":0,"internal_metadata":76349440},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:30:20 2020","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":1.028},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245980,"kb_used_data":63828,"kb_used_omap":0,"kb_used_meta":182144,"kb_avail":94125860,"statfs":{"total":96636764160,"available":96384880640,"internally_reserved":0,"allocated":65359872,"data_stored":68036638,"data_compressed":4327422,"data_compressed_allocated":12574720,"data_compressed_original":25131757,"omap_allocated":0,"internal_metadata":186515456},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:30:58 2020","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.378}]}]}]}} 2020-11-16T22:31:42.049 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:31:42.050 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [1, 2, 0] 2020-11-16T22:31:42.051 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '1', '2', '0'] 2020-11-16T22:31:42.051 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 1 2 0 2020-11-16T22:31:42.051 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:31:42.052 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:42.052 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:42.061+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:42.147 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:42.161+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:42.189 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:42.189 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:42.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.189 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30223 4.0 4:0caefcac:::1000000188a.00000001:head [trimtrunc 2150@3839578] snapc 1=[] ondisk+write+known_if_redirected+full_force e156)", 2020-11-16T22:31:42.189 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:42.200357+0000", 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0036479220000000001, 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036652970000000001, 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200357+0000", 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200357+0000", 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7542000000000002e-05 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200385+0000", 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0969999999999996e-06 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200390+0000", 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0981999999999999e-05 2020-11-16T22:31:42.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200401+0000", 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2959999999999998e-06 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200406+0000", 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4767000000000001e-05 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200441+0000", 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010605800000000001 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200547+0000", 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121074 2020-11-16T22:31:42.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200668+0000", 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000310816 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.197 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.198 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.198 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30224 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 2150@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e156)", 2020-11-16T22:31:42.198 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:42.200463+0000", 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0035419370000000002, 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036088209999999999, 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200463+0000", 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200463+0000", 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6116999999999998e-05 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200519+0000", 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1136e-05 2020-11-16T22:31:42.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200530+0000", 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5954e-05 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200546+0000", 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.622e-06 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200554+0000", 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200595+0000", 2020-11-16T22:31:42.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6297999999999994e-05 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200671+0000", 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8172999999999997e-05 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.200770+0000", 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002349556 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.203119+0000", 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026561050000000002 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30225 3.0 3:1cc859d9:::200.00000012:head [write 54540~1926 [fadvise_dontneed] in=1926b] snapc 0=[] ondisk+write+known_if_redirected+full_force e156)", 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:42.202414+0000", 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0015911790000000001, 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001701143, 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:42.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202414+0000", 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202414+0000", 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2749000000000004e-05 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202487+0000", 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9400000000000008e-06 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202496+0000", 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6152999999999999e-05 2020-11-16T22:31:42.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202512+0000", 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9450000000000004e-06 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202519+0000", 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0108999999999998e-05 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202559+0000", 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7491999999999994e-05 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202656+0000", 2020-11-16T22:31:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000157393 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:42.202814+0000", 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039978099999999999 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 3 2020-11-16T22:31:42.216 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:42.217 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:42.248 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:42.262+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:42.349 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:42.363+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:42.382 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: you must specify pairs of osd ids to be remapped 2020-11-16T22:31:42.385 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-16T22:31:42.385 INFO:tasks.thrashosds.thrasher:Failed to rm-pg-upmap-items, ignoring 2020-11-16T22:31:42.391 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:42.392 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:42.392 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:42.392 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:42.392 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:42.402 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:42.450 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:42.464+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:42.551 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:42.565+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:42.552 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:42.552 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 104.565470375, 2020-11-16T22:31:42.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:42.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:42.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:42.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:42.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.357320208, 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:42.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:42.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:42.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:42.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:42.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.48160912500001, 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:42.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:42.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:42.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.610954161, 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:42.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:42.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.826237426999995, 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.594 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:42.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:42.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:42.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.276604352999996, 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:42.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:42.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.615 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.276508913000001, 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.616 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:42.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:42.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:42.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.273974581000004, 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:42.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:42.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:42.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:42.637 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.923554788000004, 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:42.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:42.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:42.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:42.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.553868898000005, 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.648 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:42.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:42.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:42.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.658 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.553134674999995, 2020-11-16T22:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:42.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:42.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:42.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.669 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 70.491255436000003, 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:42.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:42.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:42.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 69.221159951000004, 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.681 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:42.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:42.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:42.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.691 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.480649071000002, 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:42.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:42.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:42.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:42.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.701 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.021702613000002, 2020-11-16T22:31:42.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:42.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:42.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:42.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:42.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.339631632, 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:42.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:42.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:42.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.339185759000003, 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:42.724 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:42.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:42.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.732 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.4279887, 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:42.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:42.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:42.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.742 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.647530880000001, 2020-11-16T22:31:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:31:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:31:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:31:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:31:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.54089196, 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:31:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:31:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:31:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:31:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:42.766 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:42.767 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:42.781+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:42.767 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:42.867 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:42.881+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:42.910 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:42.910 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:50853 4.5 e156/117 4:a611fa68:::1000000188a.00000002:head v 156'3590, mlcod=156'3588)", 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:42.920307+0000", 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0051372350000000004, 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0051569729999999996, 2020-11-16T22:31:42.911 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920307+0000", 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:42.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920307+0000", 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5884999999999996e-05 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920383+0000", 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4739999999999994e-06 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920392+0000", 2020-11-16T22:31:42.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4319e-05 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920407+0000", 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6780000000000003e-06 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920413+0000", 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5516000000000001e-05 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:42.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920479+0000", 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5049999999999999e-05 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:42.920504+0000", 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00019692200000000001 2020-11-16T22:31:42.917 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:31:42.918 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:42.921 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:42.968 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:42.982+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:43.057 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:43.058 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:43.058 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:43.058 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:43.058 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:43.069 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:43.070 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:43.083+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:43.169 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:43.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:43.212 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:43.212 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:43.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.90509477000001, 2020-11-16T22:31:43.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:43.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:43.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.220 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:43.221 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.222 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.90459188200001, 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:43.223 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.224 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.225 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:43.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:43.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:43.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.232 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 206.97592065000001, 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:43.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:43.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:43.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.243 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 206.25863417900001, 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.244 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:43.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:43.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:43.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:43.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.253 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01348055, 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:43.254 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:43.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:43.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:43.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.264 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01340543000001, 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:43.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:43.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:43.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.274 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01332649599999, 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:43.275 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:43.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:43.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:43.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.285 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01324907399999, 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:43.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:43.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:43.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.295 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.013168912, 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.296 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:43.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:43.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:43.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:43.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.306 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01308709899999, 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:43.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:43.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:43.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:43.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01299371299999, 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.317 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:43.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:43.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:43.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01287331699999, 2020-11-16T22:31:43.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:43.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:43.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:43.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:43.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.337 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 204.01277325000001, 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:43.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:43.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:43.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:43.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.347 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 162.61250026600001, 2020-11-16T22:31:43.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:43.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:43.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:43.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:43.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.358 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 126.766527333, 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.315725192, 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:43.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:43.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:43.377 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 104.017348133, 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:43.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:43.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:43.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:43.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:43.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:43.388 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:43.389 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 43.140400118999999, 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:43.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:43.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 38.087802132999997, 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:43.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:43.405 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:43.406 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 38.086758807000002, 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:43.407 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:43.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:43.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:43.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:43.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:43.414 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:43.416 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:43.416 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:43.430+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:43.516 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:43.530+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:43.553 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:30434 3.0 3:1cc859d9:::200.00000012:head [write 313978~1926 [fadvise_dontneed] in=1926b] snapc 0=[] ondisk+write+known_if_redirected+full_force e156)", 2020-11-16T22:31:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:43.567137+0000", 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0010556700000000001, 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001074789, 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567137+0000", 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567137+0000", 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7150000000000002e-05 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567174+0000", 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9229999999999996e-06 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567181+0000", 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1749999999999999e-05 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567193+0000", 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4759999999999997e-06 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567199+0000", 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4053000000000005e-05 2020-11-16T22:31:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567263+0000", 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3006e-05 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567326+0000", 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8416000000000001e-05 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:43.567424+0000", 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000286774 2020-11-16T22:31:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:31:43.561 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:43.562 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:43.616 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:43.631+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:43.717 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:43.731+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:43.732 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:43.749 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:43.750 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:43.750 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:43.750 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:43.750 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:43.760 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:43.818 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:43.832+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:43.877 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:43.898 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:43.898 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:43.898 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:43.898 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:43.899 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:43.900 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:43.900 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:43.919 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:43.933+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:43.939 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:43.939 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:43.949 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:44.020 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:44.034+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:44.057 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:31:44.083 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.083 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.094 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:44.120 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.135+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.221 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.235+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.285 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.285 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.296 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:44.340 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:44.355+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:44.423 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.437+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.439 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.439 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.454 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:44.524 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:44.539+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:44.593 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.594 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.603 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:44.647 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.661+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.726 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:44.741+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:44.740 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.741 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.750 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:44.834 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.848+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.929 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:44.943+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:44.939 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:44.939 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:44.950 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:45.029 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.043+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:31:45.096 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:45.096 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:45.107 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:45.148 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.163+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2020-11-16T22:31:45.230 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.245+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.291 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:45.291 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:45.302 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:45.331 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:45.345+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:45.432 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:45.446+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:45.486 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:45.486 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:45.498 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:45.533 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.547+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.631 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:45.631 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:45.633 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.647+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.641 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:45.734 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.748+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:45.774 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:45.775 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:45.978 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:45.980 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:45.981 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:45.993+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.079 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:46.094+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:46.127 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:46.128 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:46.138 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:46.148 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:46.148 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:46.148 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:46.148 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:46.150 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:46.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:46.181 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:46.196+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.278 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:46.278 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:46.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.278 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:51619 4.0 e157/153 4:0caefcac:::1000000188a.00000001:head v 157'5893, mlcod=157'5891)", 2020-11-16T22:31:46.278 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:46.291583+0000", 2020-11-16T22:31:46.279 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001473062, 2020-11-16T22:31:46.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014992600000000001, 2020-11-16T22:31:46.358 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291583+0000", 2020-11-16T22:31:46.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291583+0000", 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011689 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291700+0000", 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0137e-05 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291710+0000", 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5569999999999998e-05 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291726+0000", 2020-11-16T22:31:46.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6549999999999998e-06 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291732+0000", 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.8013000000000001e-05 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291800+0000", 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6498999999999999e-05 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:46.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:46.291817+0000", 2020-11-16T22:31:46.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00023376399999999999 2020-11-16T22:31:46.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:46.566 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:46.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:46.566 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:46.567 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:46.567 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:31:46.567 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:46.568 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:46.569 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:46.582+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.668 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:46.683+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:46.746 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:46.746 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:31:46.746 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:31:46.746 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:31:46.747 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:46.757 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:46.800 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:46.815+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.869 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:46.884+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:46.919 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:46.920 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 221.612252877, 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:46.921 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:46.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:31:46.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:46.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:31:47.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:31:47.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:31:47.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 216.611749989, 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:47.134 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:47.135 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:47.135 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:47.135 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:47.135 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:31:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:47.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:31:47.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:31:47.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:31:47.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:31:47.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:31:47.547 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.548 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.549 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 210.683078757, 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:47.550 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:47.551 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.551 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:47.551 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:47.551 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:47.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:47.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:31:47.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:31:47.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.758 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:31:47.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:31:47.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:47.762 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 209.96579228600001, 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:31:47.763 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:47.764 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:47.764 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:47.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:47.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:47.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:47.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:31:47.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:31:47.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:47.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:47.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:48.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:31:48.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:31:48.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:31:48.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:31:48.177 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:48.178 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72063865699999, 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:31:48.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:48.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:31:48.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:31:48.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:48.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:31:48.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:48.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:31:48.390 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.391 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:48.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.594 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:48.594 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:48.594 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.720563537, 2020-11-16T22:31:48.594 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:48.595 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.596 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:31:48.597 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.598 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:48.599 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.600 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:48.601 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:48.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:31:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:31:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:48.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72048460299999, 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:48.807 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:31:48.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:48.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:48.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:31:49.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:31:49.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:31:49.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:31:49.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:31:49.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:49.020 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72040718100001, 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:49.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:31:49.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:49.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:31:49.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:31:49.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.226 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:31:49.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:31:49.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:49.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:31:49.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:31:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:31:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:49.436 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.720327019, 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:49.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:31:49.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:31:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:31:49.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:49.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:31:49.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:49.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:31:49.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:49.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:31:49.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:31:49.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:49.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:31:49.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72024520599999, 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:49.853 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:49.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:49.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:49.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:31:49.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:31:49.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:50.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:31:50.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:31:50.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:50.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:31:50.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:31:50.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:31:50.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:50.269 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72015182000001, 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:50.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:50.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:50.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:50.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:50.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:31:50.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:31:50.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:31:50.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:50.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:31:50.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:31:50.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:31:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:31:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:50.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:31:50.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:50.685 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.72003142400001, 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:31:50.686 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:50.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:50.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:50.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:31:50.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:31:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:50.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:31:51.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:31:51.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:51.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:31:51.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.103 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:31:51.103 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:51.103 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 207.71993135700001, 2020-11-16T22:31:51.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:31:51.306 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:51.306 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:51.306 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:51.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:51.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:31:51.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:31:51.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:31:51.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:31:51.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:31:51.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:51.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:31:51.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:51.517 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:31:51.518 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.519 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:51.519 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.519 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:31:51.722 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:51.722 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 166.31965837300001, 2020-11-16T22:31:51.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:31:51.722 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:51.722 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:31:51.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:31:51.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:31:51.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:31:51.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:51.731 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 130.47368544, 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:51.732 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:31:51.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:51.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:31:51.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:31:51.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:51.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:31:51.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:51.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:31:51.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 108.022883299, 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:51.943 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:31:51.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:31:51.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:31:51.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:31:51.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:51.952 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 107.72450624, 2020-11-16T22:31:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:31:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:52.154 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:52.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:31:52.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:52.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:31:52.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:31:52.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:31:52.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:31:52.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:52.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.367 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 46.847558225999997, 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:31:52.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:31:52.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:31:52.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.375 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.794960240000002, 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:52.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:31:52.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:31:52.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:31:52.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.793916914, 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:31:52.384 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:31:52.385 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:52.385 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:31:52.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.586 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:52.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:52.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:52.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:31:52.588 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.589 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:52.590 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:52.591 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:52.592 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:31:52.592 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:31:52.592 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:31:52.592 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.593 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:52.593 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.593 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:52.593 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:52.593 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:52.594 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:31:52.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:31:52.595 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:31:52.595 INFO:tasks.thrashosds.thrasher:inject_pause on 2 2020-11-16T22:31:52.595 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:31:52.595 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:31:52.596 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2020-11-16T22:31:52.596 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:52.598 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:52.598 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:52.609+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:52.695 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:52.709+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:52.739 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:52.739 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-11-16T22:31:52.739 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:52.740 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:52.740 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:53144 4.5 4:a611fa68:::1000000188a.00000002:head [write 1986560~8994 [2389@0] in=8994b] snapc 1=[] ondisk+write+known_if_redirected e157)", 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:52.752276+0000", 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.003223358, 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003245501, 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:52.741 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 53144 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752276+0000", 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752276+0000", 2020-11-16T22:31:52.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9089000000000001e-05 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752355+0000", 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5580000000000006e-06 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752365+0000", 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6464999999999999e-05 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752381+0000", 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3869999999999997e-06 2020-11-16T22:31:52.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752389+0000", 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6959999999999998e-05 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752426+0000", 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7397999999999994e-05 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752493+0000", 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105188 2020-11-16T22:31:52.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:52.752598+0000", 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032204500000000002 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:52.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:52.749 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:52.749 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:31:52.749 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:31:52.749 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:52.751 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:52.769 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:52.769 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:52.770 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:52.770 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:52.772 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:52.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:52.795 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:52.810+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:52.896 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:52.911+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:52.935 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:52.935 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:52.945 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:52.997 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.012+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.073 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:31:53.098 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.113+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.106 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.107 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.118 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:53.199 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.214+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.256 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.257 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.267 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:53.300 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.315+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.401 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:53.416+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:53.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2020-11-16T22:31:53.404 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.404 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.415 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:53.502 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:53.516+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:53.601 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.602 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.604 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:53.618+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:53.612 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:53.705 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.720+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.757 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.757 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.769 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:53.809 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.824+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.907 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:53.921+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:53.954 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:53.954 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:53.965 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:54.008 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.023+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.109 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.123+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.145 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:54.145 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:54.156 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:54.209 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.224+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.307 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:54.307 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:54.310 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.325+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.318 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:54.410 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.425+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.508 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:54.509 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:54.511 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:54.526+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:54.520 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:54.611 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.626+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.704 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:54.705 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:54.713 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.727+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.716 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:54.773 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:31:54.813 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.828+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.853 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:54.853 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:54.864 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:54.914 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:54.929+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:54.921 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:54.941 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:54.942 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:54.942 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:54.942 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:31:54.942 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:31:54.944 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:54.944 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:55.009 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:55.009 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:31:55.009 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:31:55.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:31:55.010 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:55.015 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:55.030+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:55.019 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:31:55.085 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:31:55.115 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:55.131+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 117.21012755700001, 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:55.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:55.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:55.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:31:55.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:31:55.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.206 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 116.00197738999999, 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:31:55.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:55.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:55.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:31:55.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:31:55.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:31:55.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:31:55.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:55.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:31:55.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:31:55.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:31:55.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:55.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 115.12626630699999, 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:31:55.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:55.426 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:55.426 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:55.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:31:55.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:55.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:31:55.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:55.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:31:55.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:31:55.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:55.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.839 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(4.7 110/109 [PushOp(4:e4e5e45f:::10000000897.00000000:head, version: 87'211, data_included: [0~177], data_size: 177, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4e5e45f:::10000000897.00000000:head@87'211, size: 177, copy_subset: [0~177], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:177, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 114.255611343, 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773, 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954584+0000", 2020-11-16T22:31:55.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7781e-05 2020-11-16T22:31:55.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:55.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:55.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:55.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954612+0000", 2020-11-16T22:31:55.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6420000000000002e-06 2020-11-16T22:31:56.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:56.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954622+0000", 2020-11-16T22:31:56.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6668999999999999e-05 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954638+0000", 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954645+0000", 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0727999999999999e-05 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954706+0000", 2020-11-16T22:31:56.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4590000000000004e-06 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.954716+0000", 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014806231 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.969522+0000", 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01493773 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.051 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18468 3.0 3:1e8917d7:::200.0000000e:head [write 1676814~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:56.254 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:56.254 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.470894608999998, 2020-11-16T22:31:56.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903, 2020-11-16T22:31:56.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:56.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.739301+0000", 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010492105999999999 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749793+0000", 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6376e-05 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749819+0000", 2020-11-16T22:31:56.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5550000000000008e-06 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749829+0000", 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:56.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749833+0000", 2020-11-16T22:31:56.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3223999999999998e-05 2020-11-16T22:31:56.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:56.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749876+0000", 2020-11-16T22:31:56.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7378000000000003e-05 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.749964+0000", 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142984 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.750107+0000", 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019353650000000001 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.752042+0000", 2020-11-16T22:31:56.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019812129999999999 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754023+0000", 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9922000000000001e-05 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754073+0000", 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7436e-05 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:56.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:29.754101+0000", 2020-11-16T22:31:56.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014799903 2020-11-16T22:31:56.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.467 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:56.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:56.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:56.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.921261534999999, 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:56.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:31:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:31:56.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:31:56.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:56.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:31:56.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:31:56.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:31:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:31:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.921166095000004, 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:56.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:31:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.918631762999993, 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:31:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:31:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:31:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:31:57.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:31:57.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:31:57.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.105 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.568211969999993, 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:31:57.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:57.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:31:57.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:31:57.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:31:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:31:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:31:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.198526079999993, 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:31:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:31:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:31:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:31:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:31:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.197791856999999, 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:31:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:31:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:31:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:57.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:31:57.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.135912618000006, 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:57.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:57.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:31:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.865817132999993, 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:31:57.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:31:57.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:57.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:57.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:31:57.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:31:57.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:57.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.125306252999998, 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:57.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:57.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:57.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:57.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:57.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:57.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:31:58.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:31:58.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:31:58.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:31:58.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 52.666359794999998, 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:58.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:31:58.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:31:58.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:31:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:31:58.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.984288814000003, 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:58.390 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:31:58.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:31:58.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:31:58.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:31:58.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:31:58.399 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.399 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:31:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.983842940999999, 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.072645882000003, 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:58.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:58.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:58.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:31:58.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:31:58.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:31:58.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:31:58.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:31:58.823 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.292188062000001, 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:58.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:31:58.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:31:58.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:58.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 24.185549141999999, 2020-11-16T22:31:58.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:31:58.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:31:59.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:31:59.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:31:59.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:31:59.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:31:59.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:31:59.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:31:59.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:31:59.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:31:59.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:31:59.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:31:59.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:31:59.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:31:59.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:31:59.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:31:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:31:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:31:59.252 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:31:59.253 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:31:59.253 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:31:59.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:31:59.254 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:31:59.254 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2020-11-16T22:31:59.256 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:31:59.256 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:59.267+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:59.257 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:31:59.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:59.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:32552 3.0 e159/111 3:1cc859d9:::200.00000012:head v 159'10205, mlcod=159'10203)", 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:59.414532+0000", 2020-11-16T22:31:59.401 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.001333639, 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001349414, 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:31:59.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414532+0000", 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414532+0000", 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2790999999999997e-05 2020-11-16T22:31:59.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414575+0000", 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0399999999999998e-06 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:31:59.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414581+0000", 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.91e-06 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414591+0000", 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3869999999999997e-06 2020-11-16T22:31:59.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414595+0000", 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5985000000000002e-05 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414631+0000", 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5704000000000001e-05 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:59.414647+0000", 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114817 2020-11-16T22:31:59.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:31:59.408 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:31:59.411 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:31:59.454 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:31:59.475 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:31:59.475 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:31:59.476 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:31:59.476 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:31:59.476 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:31:59.478 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:31:59.478 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:31:59.493 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:59.508+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:59.554 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:59.569+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:31:59.591 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:59.591 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:59.602 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:31:59.685 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:59.700+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:59.756 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:59.771+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:59.777 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:59.777 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:59.787 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:31:59.857 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:31:59.872+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:31:59.924 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:31:59.925 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:31:59.934 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:31:59.977 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:31:59.992+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.059 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.073+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~42s 2020-11-16T22:32:00.090 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.090 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:moved 9 / 50 (18%) 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 4.12311 -> 8.51469 (expected baseline 3.53553) 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 20 -> 17 pgs (0.8 -> 0.68 * mean) 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.2 with 22 -> 16 pgs (0.88 -> 0.64 * mean) 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:32:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 3 2020-11-16T22:32:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0019 2020-11-16T22:32:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0193 2020-11-16T22:32:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 weight 0.5670 -> 0.9388 2020-11-16T22:32:00.101 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:00.111 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:00.111 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2020-11-16T22:32:00.160 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:00.174+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:00.260 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.275+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.264 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.265 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.274 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:00.361 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.376+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.400 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.400 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.409 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:00.462 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:00.477+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:00.533 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.533 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.542 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:00.563 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.578+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.664 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.678+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.707 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.707 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.716 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:00.764 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.779+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:00.846 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:00.847 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:00.855 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:00.865 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:00.880+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:00.966 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:00.981+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:01.018 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:01.019 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:01.028 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:01.067 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.097 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:32:01.097 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:01.097 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 8.51469 -> 8.51469 (expected baseline 3.53553) 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 17 -> 17 pgs (0.68 -> 0.68 * mean) 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:01.098 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0019 2020-11-16T22:32:01.099 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0021 2020-11-16T22:32:01.099 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 1.0000 -> 0.9500 2020-11-16T22:32:01.099 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 weight 0.9388 -> 0.9888 2020-11-16T22:32:01.112 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:01.112 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2020-11-16T22:32:01.168 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.183+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.194 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:01.194 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:01.203 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:01.257 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:32:01.270 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.284+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.329 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:01.329 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:01.338 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:01.371 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:01.385+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:01.472 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:01.486+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:01.479 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:01.503 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:01.503 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:01.511 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:01.573 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.587+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.622 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:01.639 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:01.640 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 236.332540924, 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:01.641 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 231.332038036, 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:32:01.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:32:01.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.658 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 225.403366804, 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:32:01.659 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:32:01.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.668 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 224.68608033300001, 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:32:01.669 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:32:01.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:32:01.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:32:01.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.678 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44092670399999, 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.679 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:32:01.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:32:01.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.684 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:32:01.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:32:01.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.440851584, 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.689 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:32:01.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:32:01.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:32:01.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:32:01.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44077265000001, 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.699 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:32:01.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:32:01.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44069522800001, 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.709 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:32:01.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44061506599999, 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:32:01.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:32:01.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:32:01.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44053325300001, 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.729 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.730 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:32:01.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:32:01.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44043986700001, 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.739 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:32:01.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:32:01.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.748 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.44031947100001, 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:01.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:32:01.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:32:01.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:32:01.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:01.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 222.440219404, 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:32:01.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:32:01.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:32:01.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:32:01.768 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 181.03994642000001, 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:01.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:32:01.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:32:01.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:32:01.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:32:01.778 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 145.19397348699999, 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:32:01.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:32:01.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.786 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.743171346, 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:32:01.787 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:32:01.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:32:01.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:01.796 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 122.44479428699999, 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:01.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:32:01.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:32:01.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:32:01.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:01.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:32:01.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:32:01.807 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.567846273000001, 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:32:01.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:32:01.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:32:01.815 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 56.515248286999999, 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:32:01.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:32:01.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:32:01.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:01.823 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 56.514204960999997, 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:32:01.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:01.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:32:01.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:32:01.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:01.831 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:01.832 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:01.833 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:01.833 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:01.833 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:01.833 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:01.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:01.834 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:01.834 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.847+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.834 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:01.932 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:01.947+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:01.962 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:01.962 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:01.962 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:32:01.963 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:01.971 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:02.033 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.047+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.094 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:02.095 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:02.095 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:32:02.095 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:32:02.095 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:02.103 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:02.105 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:32:02.106 INFO:teuthology.orchestra.run.smithi138.stdout:moved 1 / 50 (2%) 2020-11-16T22:32:02.106 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:02.106 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 8.51469 -> 9 (expected baseline 3.53553) 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.2 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 17 -> 16 pgs (0.68 -> 0.64 * mean) 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:02.107 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0019 2020-11-16T22:32:02.108 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0021 2020-11-16T22:32:02.108 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 0.9500 -> 0.9000 2020-11-16T22:32:02.108 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 weight 0.9888 -> 1.0000 2020-11-16T22:32:02.121 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:02.121 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2020-11-16T22:32:02.144 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.159+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.234 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:02.249+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:02.291 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:02.291 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 124.30427543899999, 2020-11-16T22:32:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:32:02.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:32:02.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:32:02.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 123.09612527199999, 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:32:02.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:32:02.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:02.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 122.220414189, 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:02.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.015409417000001, 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:32:02.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:32:02.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:32:02.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:32:02.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.334 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.015313977000005, 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:32:02.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:32:02.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:32:02.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:32:02.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.012779644999995, 2020-11-16T22:32:02.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:02.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:32:02.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:32:02.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:32:02.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:32:02.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:02.355 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.662359851999994, 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:02.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:32:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:32:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:32:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:32:02.364 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:02.365 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.292673961999995, 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:32:02.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:32:02.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:32:02.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:32:02.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:32:02.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.291939739, 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:32:02.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:32:02.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:32:02.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:32:02.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:32:02.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.230060499999993, 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.386 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:02.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:32:02.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:32:02.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:32:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.959965014999995, 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:32:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:32:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:32:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.219454134999999, 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:32:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.760507677, 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:32:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:32:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.078436695999997, 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:32:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:32:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:32:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:32:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.077990823, 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:32:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:32:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.166793763999998, 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:32:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:32:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:32:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.386335944000002, 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:32:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:32:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:32:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.279697024000001, 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:32:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:32:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:32:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:32:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:54217 4.5 4:a611fa68:::1000000188a.00000002:head [read 937984~24576 [2500@0]] snapc 0=[] ondisk+read+known_if_redirected e159)", 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7459555760000001, 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999, 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 54217 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023326738 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581714+0000", 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.36e-05 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581728+0000", 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3904999999999999e-05 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581752+0000", 2020-11-16T22:32:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1073999999999999e-05 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581763+0000", 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4217999999999998e-05 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581857+0000", 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013256300000000001 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581990+0000", 2020-11-16T22:32:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1364000000000002e-05 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.582071+0000", 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32400 3.0 3:1cc859d9:::200.00000012:head [write 2817827~1931 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e159)", 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.1231477600000002, 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999, 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4440999999999998e-05 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181230+0000", 2020-11-16T22:32:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4779999999999999e-06 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181235+0000", 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2499999999999995e-06 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181245+0000", 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2200000000000003e-06 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181249+0000", 2020-11-16T22:32:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6591000000000003e-05 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181286+0000", 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3828e-05 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181339+0000", 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010155800000000001 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181441+0000", 2020-11-16T22:32:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023834762999999998 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205276+0000", 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000670226 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205946+0000", 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4202999999999997e-05 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205990+0000", 2020-11-16T22:32:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6330999999999999e-05 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.206016+0000", 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:02.495 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 9 -> 9 (expected baseline 3.53553) 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0019 2020-11-16T22:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0194 2020-11-16T22:32:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:32:02.498 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:32:02.498 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2020-11-16T22:32:02.499 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.510+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.499 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:02.597 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.611+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.648 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:02.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:54935 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 3856032~20528 [2568@4194304] in=20528b] snapc 1=[] ondisk+write+known_if_redirected e162)", 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:02.658774+0000", 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0044484820000000001, 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0044678019999999999, 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:02.649 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 54935 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:02.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658774+0000", 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658774+0000", 2020-11-16T22:32:02.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5258e-05 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658810+0000", 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0100000000000003e-06 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658815+0000", 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1328000000000001e-05 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658826+0000", 2020-11-16T22:32:02.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5110000000000003e-06 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658832+0000", 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5148000000000002e-05 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658877+0000", 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8702000000000005e-05 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.658975+0000", 2020-11-16T22:32:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017691399999999999 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659152+0000", 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003662733 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.662815+0000", 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040406039999999997 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:54936 4.0 e162/161 4:0caefcac:::1000000188a.00000001:head v 162'6590, mlcod=162'6588)", 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:02.659326+0000", 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0038970440000000001, 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039685900000000001, 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:02.658 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659326+0000", 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659326+0000", 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2351000000000001e-05 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659378+0000", 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.378e-06 2020-11-16T22:32:02.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659384+0000", 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2050000000000008e-06 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:02.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659393+0000", 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0459999999999999e-06 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659397+0000", 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3715999999999999e-05 2020-11-16T22:32:02.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659451+0000", 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2632999999999997e-05 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:02.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:02.659483+0000", 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000157329 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:32:02.664 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:32:02.665 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:02.665 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:02.709 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:02.724+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:02.798 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.813+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.805 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:32:02.805 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:02.805 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 9 -> 9 (expected baseline 3.53553) 2020-11-16T22:32:02.805 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:02.805 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 16 -> 16 pgs (0.64 -> 0.64 * mean) 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0019 2020-11-16T22:32:02.806 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0194 2020-11-16T22:32:02.807 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:32:02.812 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:32:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:32:02.813 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:32:02.813 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:02.823 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:02.899 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:02.913+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:02.999 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:03.014+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:03.008 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:03.008 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 237.70095755599999, 2020-11-16T22:32:03.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:32:03.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:32:03.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:32:03.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:32:03.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 232.70045466799999, 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:03.019 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:03.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:32:03.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:32:03.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:32:03.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 226.77178343599999, 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:32:03.028 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.037 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 226.054496965, 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:32:03.038 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.047 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.80934333600001, 2020-11-16T22:32:03.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:32:03.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:32:03.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.057 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.80926821599999, 2020-11-16T22:32:03.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:03.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:32:03.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.062 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:32:03.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:32:03.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:32:03.067 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.068 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.80918928200001, 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:03.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:32:03.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:32:03.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.078 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.80911186, 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:32:03.079 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:32:03.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:32:03.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:32:03.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.089 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.80903169800001, 2020-11-16T22:32:03.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:03.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:32:03.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:32:03.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:32:03.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.100 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:03.101 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.808949885, 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:32:03.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:32:03.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:32:03.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:32:03.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.112 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.808856499, 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:32:03.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:32:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:32:03.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:32:03.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.123 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.808736103, 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.124 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.125 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:32:03.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:32:03.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:32:03.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.808636036, 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.135 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:32:03.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:32:03.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:32:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.145 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 182.408363052, 2020-11-16T22:32:03.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:03.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:32:03.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:32:03.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:32:03.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:32:03.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 146.56239011900001, 2020-11-16T22:32:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:32:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:32:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:32:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:32:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:32:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 124.111587978, 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:32:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:32:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:32:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:32:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:32:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:32:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 123.813210919, 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:32:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:32:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:32:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:32:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:32:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:32:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:32:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 62.936262905, 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:32:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:32:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.883664918999997, 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:32:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:32:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.882621593000003, 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:32:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:32:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:03.218 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:03.219 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:03.233+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:03.219 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:03.319 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:03.334+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:03.366 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:03.366 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:03.366 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2020-11-16T22:32:03.367 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:03.378 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:07.647 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:07.647 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:32:07.649 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:07.662+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:07.748 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:07.762+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:07.782 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:32:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:32:07.783 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:07.793 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:07.806 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:07.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:32:07.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:32:07.819 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2020-11-16T22:32:07.819 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2020-11-16T22:32:07.827 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:07.827 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:07.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:07.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:07.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:07.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:07.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:07.848 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:07.863+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:07.935 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:07.935 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:07.945 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:07.988 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.003+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.049 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.063+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.095 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.095 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.106 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:08.150 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:08.164+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:08.250 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.264+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.289 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.289 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.299 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:08.382 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.396+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.451 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:08.466+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:08.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2020-11-16T22:32:08.479 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.480 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.491 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:08.553 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.566+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.631 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.632 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.641 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:08.685 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:08.700+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:08.754 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.767+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.788 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.788 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.798 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:08.829 INFO:tasks.ceph.mon.a.smithi138.stderr:2020-11-16T22:32:08.844+0000 7f9f33954700 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2020-11-16T22:32:08.838 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:32:08.838 INFO:teuthology.orchestra.run.smithi138.stderr:osd set-full-ratio 0.001 2020-11-16T22:32:08.854 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.868+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.955 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:08.968+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:08.975 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:08.976 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:08.985 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:09.055 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.069+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.125 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:09.125 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:09.136 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:09.176 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.190+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.257 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.270+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.322 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:09.322 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:09.332 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:09.358 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:09.372+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:09.458 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.472+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.523 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:09.523 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:09.534 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:09.578 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:09.592+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:09.661 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.674+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.672 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:09.672 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:09.682 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:09.761 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.775+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.832 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:09.851 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2020-11-16T22:32:09.852 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2020-11-16T22:32:09.858 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:09.858 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:09.862 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.876+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.869 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:09.963 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:09.977+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:09.984 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:10.007 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:10.007 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:10.007 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:10.007 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:10.007 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:10.009 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:10.010 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:10.018 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:10.018 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:32:10.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:32:10.018 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:10.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 132.03120354500001, 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:10.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:32:10.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:32:10.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:32:10.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:10.028 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 130.823053378, 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:32:10.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:32:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:32:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:32:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:32:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:32:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:10.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:32:10.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:10.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:32:10.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:10.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:32:10.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:32:10.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGPush(3.7 110/109 [PushOp(3:f83d65d8:::200.00000004:head, version: 88'8, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f83d65d8:::200.00000004:head@88'8, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 129.947342295, 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998, 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.083929+0000", 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001891333 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085820+0000", 2020-11-16T22:32:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9422000000000001e-05 2020-11-16T22:32:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:10.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085850+0000", 2020-11-16T22:32:10.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8081000000000001e-05 2020-11-16T22:32:10.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085878+0000", 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2975e-05 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085891+0000", 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9751000000000002e-05 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:10.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085981+0000", 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2427e-05 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.085993+0000", 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014715816 2020-11-16T22:32:10.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.100709+0000", 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.016779804999999998 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:10.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.742337523000003, 2020-11-16T22:32:10.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:32:11.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:11.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:11.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:11.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:11.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:32:11.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:11.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:32:11.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:11.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:32:11.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:32:11.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:11.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:11.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:11.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.742242082999994, 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:11.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:11.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:11.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:32:11.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:11.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:32:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:32:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:32:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:32:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:32:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:32:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:32:11.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:11.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:32:11.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:11.899 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.739707750999997, 2020-11-16T22:32:11.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:11.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:11.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:11.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:11.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:32:11.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:32:12.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:12.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:32:12.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:32:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:32:12.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:12.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:32:12.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:32:12.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:12.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:32:12.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:12.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.389287957999997, 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:12.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:12.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:32:12.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:32:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:32:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:32:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:12.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:32:12.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:32:12.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:12.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:32:12.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:12.733 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.019602067999998, 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:12.734 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:32:12.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:12.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:12.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:12.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:12.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:32:12.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:12.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:32:12.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:32:12.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:12.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:12.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:13.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:32:13.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:32:13.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:32:13.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:13.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:13.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:13.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.018867845000003, 2020-11-16T22:32:13.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:32:13.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:13.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:13.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:32:13.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:13.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:32:13.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:32:13.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:32:13.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:13.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.770 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:13.770 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:13.770 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 97.956988605999996, 2020-11-16T22:32:13.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:32:13.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:32:13.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:13.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:32:13.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:13.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:32:13.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:32:13.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:13.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:32:13.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:13.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:32:13.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:32:13.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:13.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:13.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:13.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:32:13.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:32:14.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:14.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 96.686893120999997, 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:14.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:14.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:32:14.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:32:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:32:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:32:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:32:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 69.946382240999995, 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:32:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:32:14.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:14.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:32:14.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:32:14.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:14.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:14.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:14.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:32:14.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:32:14.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:32:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:32:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:15.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.487435782999995, 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:32:15.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:15.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:32:15.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:15.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:32:15.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:32:15.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:32:15.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:15.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:32:15.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:32:15.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:32:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:32:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:15.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:32:15.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:15.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.805364802, 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:32:15.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:15.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:15.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:15.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:15.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:15.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:15.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:15.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:32:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:32:15.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:32:15.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:15.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:15.855 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:15.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.804918928999996, 2020-11-16T22:32:16.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:32:16.058 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:16.058 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.059 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:32:16.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:32:16.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:16.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:32:16.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:32:16.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:32:16.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:16.268 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.893721869999993, 2020-11-16T22:32:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:16.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:16.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:16.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:32:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:32:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:32:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:32:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:32:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:32:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.113264049999998, 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:32:16.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.890 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:16.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:16.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:16.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:16.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:32:16.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:16.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:16.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:16.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:32:16.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:32:16.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:32:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.006625130000003, 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:32:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:17.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:17.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:17.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:17.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:32:17.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:17.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:32:17.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:32:17.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:32:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:32:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:32:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:32:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:32:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:32:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:32:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:54217 4.5 4:a611fa68:::1000000188a.00000002:head [read 937984~24576 [2500@0]] snapc 0=[] ondisk+read+known_if_redirected e159)", 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.472883682000001, 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999, 2020-11-16T22:32:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 54217 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.724 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023326738 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581714+0000", 2020-11-16T22:32:17.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.36e-05 2020-11-16T22:32:17.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:17.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581728+0000", 2020-11-16T22:32:17.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3904999999999999e-05 2020-11-16T22:32:17.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581752+0000", 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1073999999999999e-05 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581763+0000", 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4217999999999998e-05 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581857+0000", 2020-11-16T22:32:17.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013256300000000001 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581990+0000", 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1364000000000002e-05 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.582071+0000", 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:17.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:17.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:17.935 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32400 3.0 3:1cc859d9:::200.00000012:head [write 2817827~1931 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e159)", 2020-11-16T22:32:17.935 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:17.935 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 11.850075865999999, 2020-11-16T22:32:17.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999, 2020-11-16T22:32:18.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:18.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:18.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:18.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:18.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4440999999999998e-05 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181230+0000", 2020-11-16T22:32:18.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4779999999999999e-06 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181235+0000", 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2499999999999995e-06 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181245+0000", 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2200000000000003e-06 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:18.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181249+0000", 2020-11-16T22:32:18.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6591000000000003e-05 2020-11-16T22:32:18.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:18.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181286+0000", 2020-11-16T22:32:18.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3828e-05 2020-11-16T22:32:18.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181339+0000", 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010155800000000001 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181441+0000", 2020-11-16T22:32:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023834762999999998 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205276+0000", 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000670226 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205946+0000", 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4202999999999997e-05 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:18.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205990+0000", 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6330999999999999e-05 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.206016+0000", 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999 2020-11-16T22:32:18.350 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:18.351 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:18.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:18.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:18.351 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:18.351 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:18.554 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:32:18.555 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:18.558 INFO:teuthology.orchestra.run.smithi138.stderr:osd set-full-ratio 0.95 2020-11-16T22:32:18.558 INFO:tasks.workunit.client.0.smithi138.stdout:50000 mapread 0x648404 thru 0x648abe (0x6bb bytes) 2020-11-16T22:32:18.558 INFO:tasks.workunit.client.0.smithi138.stdout:All 50000 operations completed A-OK! 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:Seed set to 1 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:skipping zero size read 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x7e0023 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:fallocating to largest ever: 0x16e26fa 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x51314a1 2020-11-16T22:32:18.559 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x529f232 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x53b012c 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:fallocating to largest ever: 0x548ef5b 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x623914d 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x628f369 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:copying to largest ever: 0x6328dee 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x63b535e 2020-11-16T22:32:18.560 INFO:tasks.workunit.client.0.smithi138.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2020-11-16T22:32:18.561 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2020-11-16T22:32:18.561 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2020-11-16T22:32:18.561 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support clone range, disabling! 2020-11-16T22:32:18.561 INFO:tasks.workunit.client.0.smithi138.stderr:main: filesystem does not support dedupe range, disabling! 2020-11-16T22:32:18.562 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:18.568+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:18.562 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:18.656 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:18.670+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:18.710 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:18.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:32:18.710 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:32:18.710 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:18.720 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:18.750 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:18.762 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:18.777+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:18.967 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:18.967 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:18.968 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:18.968 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:18.968 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:18.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:18.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:18.969 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:18.969 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:18.969 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:18.970 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:18.981+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.067 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:19.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.152 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:19.152 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:19.164 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:19.205 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:19.220+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.270 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:19.284+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:19.305 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:19.306 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:19.319 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:19.403 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:19.418+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:19.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~61s 2020-11-16T22:32:19.471 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:19.485+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.510 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:19.510 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:19.522 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:19.571 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:19.586+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.672 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:19.686+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:19.711 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:19.711 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:19.721 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:19.773 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:19.787+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:19.869 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:19.869 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:19.873 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:19.888+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:19.879 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:19.937 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:32:19.974 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:19.988+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.024 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.025 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.036 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:20.080 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:20.095+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:20.177 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:20.191+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.184 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.185 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.196 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:20.278 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:20.292+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.342 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.342 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.352 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:20.379 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:20.394+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:20.479 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:20.494+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.539 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.540 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.550 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:20.591 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:20.606+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.681 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:20.696+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:20.731 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.731 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.741 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:20.784 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:20.799+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:20.883 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:20.898+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:20.884 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:20.885 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:20.895 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:20.969 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:20.984 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:20.999+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:21.080 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:21.080 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:21.085 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:21.100+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:21.090 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:21.127 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:21.150 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:21.151 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:21.151 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:21.151 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:21.153 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:21.154 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:21.186 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:21.200+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:21.279 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:21.279 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 255.97163273199999, 2020-11-16T22:32:21.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:21.281 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:21.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:21.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:21.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:21.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:32:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:21.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:32:21.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:32:21.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:32:21.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:21.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:32:21.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:32:21.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:21.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:32:21.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:32:21.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:21.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:32:21.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:32:21.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:21.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:32:21.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:32:21.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:21.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:32:21.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:32:21.702 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:21.702 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:21.703 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:21.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.704 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:21.704 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:21.704 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 250.97112984399999, 2020-11-16T22:32:21.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:32:21.705 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:21.705 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:21.906 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:21.906 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:21.906 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:21.906 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:21.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:32:21.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:32:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:21.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:32:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:32:22.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:32:22.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.117 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:141 3.3 3:d5c7a900:::200.00000003:head [write 1036370~3157934 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 245.04245861199999, 2020-11-16T22:32:22.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594, 2020-11-16T22:32:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:22.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:22.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:22.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:22.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.249751+0000", 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0038272380000000002 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253578+0000", 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6470000000000003e-06 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253583+0000", 2020-11-16T22:32:22.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2204e-05 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253596+0000", 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.219e-06 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253599+0000", 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5499e-05 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:22.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253634+0000", 2020-11-16T22:32:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4641000000000007e-05 2020-11-16T22:32:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:22.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253699+0000", 2020-11-16T22:32:22.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012218299999999999 2020-11-16T22:32:22.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.253821+0000", 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024314120000000002 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:22.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.278135+0000", 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001350361 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279486+0000", 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2230999999999993e-05 2020-11-16T22:32:22.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279558+0000", 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000115251 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.279673+0000", 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029922594 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.534 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:22.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:22.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 244.325172141, 2020-11-16T22:32:22.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:32:22.738 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:22.738 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:22.738 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:22.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:22.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:22.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:32:22.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:22.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:32:22.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:32:22.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:22.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:32:22.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:32:22.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:32:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:32:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:32:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:32:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:22.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:32:22.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:32:22.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:22.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:32:22.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:32:22.952 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.953 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:22.953 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:22.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:22.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:22.953 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1108 3.6 3:7bf7e7b5:::200.00000006:head [write 1371633~1386 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:23.158 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:23.158 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.08001851200001, 2020-11-16T22:32:23.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001, 2020-11-16T22:32:23.158 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:23.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212191+0000", 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6674999999999997e-05 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212227+0000", 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3440000000000003e-06 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:23.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212232+0000", 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6579999999999996e-06 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212238+0000", 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5670000000000002e-06 2020-11-16T22:32:23.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:23.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212241+0000", 2020-11-16T22:32:23.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020310050999999999 2020-11-16T22:32:23.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232551+0000", 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0114e-05 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:23.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232621+0000", 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000116104 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.232737+0000", 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0026061270000000002 2020-11-16T22:32:23.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235343+0000", 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0036851509999999998 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239028+0000", 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7754999999999999e-05 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239076+0000", 2020-11-16T22:32:23.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0809000000000005e-05 2020-11-16T22:32:23.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:23.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239157+0000", 2020-11-16T22:32:23.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.026966355000000001 2020-11-16T22:32:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:23.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1109 3.6 3:7bf7e7b5:::200.00000006:head [write 1373019~1877 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.07994339199999, 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999, 2020-11-16T22:32:23.575 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:23.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212266+0000", 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7104999999999998e-05 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212303+0000", 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1300000000000003e-06 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212307+0000", 2020-11-16T22:32:23.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5830000000000003e-06 2020-11-16T22:32:23.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:23.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212314+0000", 2020-11-16T22:32:23.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4139999999999998e-06 2020-11-16T22:32:23.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:23.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212317+0000", 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.020771185000000001 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233088+0000", 2020-11-16T22:32:23.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5386e-05 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233164+0000", 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119852 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233283+0000", 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034232569999999999 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236707+0000", 2020-11-16T22:32:23.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0024927439999999999 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239199+0000", 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8000999999999997e-05 2020-11-16T22:32:23.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:23.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239247+0000", 2020-11-16T22:32:23.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4318000000000002e-05 2020-11-16T22:32:23.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:23.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239322+0000", 2020-11-16T22:32:23.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027055974999999999 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1110 3.6 3:7bf7e7b5:::200.00000006:head [write 1374896~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:23.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.079864458, 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001, 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212345+0000", 2020-11-16T22:32:23.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8288999999999999e-05 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212383+0000", 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1110000000000001e-06 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:23.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:23.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:23.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212387+0000", 2020-11-16T22:32:23.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4629999999999998e-06 2020-11-16T22:32:23.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212394+0000", 2020-11-16T22:32:24.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4949999999999998e-06 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212396+0000", 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021163962000000001 2020-11-16T22:32:24.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233560+0000", 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103664 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233664+0000", 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017732499999999999 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.233841+0000", 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028758590000000001 2020-11-16T22:32:24.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236717+0000", 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0040154880000000002 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240732+0000", 2020-11-16T22:32:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1857000000000001e-05 2020-11-16T22:32:24.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:24.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240794+0000", 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5507999999999996e-05 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240870+0000", 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028525021000000001 2020-11-16T22:32:24.407 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1111 3.6 3:7bf7e7b5:::200.00000006:head [write 1376280~1384 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:24.408 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.079787036, 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555, 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212422+0000", 2020-11-16T22:32:24.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7914999999999999e-05 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212460+0000", 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9990000000000002e-06 2020-11-16T22:32:24.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212464+0000", 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3010000000000001e-06 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212471+0000", 2020-11-16T22:32:24.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.478e-06 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212475+0000", 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.021797732 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234273+0000", 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010668200000000001 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:24.617 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234379+0000", 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017341200000000001 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234553+0000", 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033684280000000001 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.618 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237921+0000", 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0029981410000000002 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:24.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240919+0000", 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8439000000000001e-05 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240978+0000", 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0028e-05 2020-11-16T22:32:24.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241058+0000", 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028635555 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1112 3.6 3:7bf7e7b5:::200.00000006:head [write 1377664~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.07970687400001, 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001, 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:24.622 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.624 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:24.624 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212502+0000", 2020-11-16T22:32:24.624 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9495e-05 2020-11-16T22:32:24.624 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.624 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:24.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212542+0000", 2020-11-16T22:32:24.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6969999999999999e-06 2020-11-16T22:32:24.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212546+0000", 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1099999999999992e-06 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:24.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212555+0000", 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.694e-06 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212557+0000", 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022352314000000002 2020-11-16T22:32:24.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234910+0000", 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0403999999999999e-05 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:24.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.234980+0000", 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118838 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235099+0000", 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0028302779999999999 2020-11-16T22:32:24.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237929+0000", 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0031670999999999999 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:24.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241096+0000", 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8683000000000001e-05 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241145+0000", 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0067000000000005e-05 2020-11-16T22:32:24.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.241215+0000", 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.028712680000000001 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:24.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.079625061, 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:24.834 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:24.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:32:25.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:32:25.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:32:25.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:32:25.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.079531675, 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:25.047 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:25.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:25.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:25.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:25.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:25.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:32:25.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:25.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:32:25.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:32:25.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:32:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:32:25.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:25.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:32:25.461 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:25.462 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.079411279, 2020-11-16T22:32:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:32:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:25.463 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:25.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:25.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:25.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:32:25.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:25.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:32:25.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:32:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:32:25.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:25.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:32:25.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:25.882 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:25.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:25.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:25.883 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 242.07931121199999, 2020-11-16T22:32:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:32:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:26.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:32:26.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:32:26.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:26.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:32:26.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:32:26.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:32:26.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:32:26.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:32:26.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:32:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:32:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:26.299 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.502 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 200.679038228, 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:26.503 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:32:26.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:32:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:32:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:32:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:32:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:32:26.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:32:26.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:32:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:32:26.918 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:14179 3.0 3:07a02702:::200.0000000c:head [write 3752002~1930 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e103)", 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 164.83306529500001, 2020-11-16T22:32:26.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999, 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:26.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459144+0000", 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4133e-05 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459178+0000", 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0409999999999997e-06 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459183+0000", 2020-11-16T22:32:26.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4349999999999997e-06 2020-11-16T22:32:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:26.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:27.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459192+0000", 2020-11-16T22:32:27.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0029999999999996e-06 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459196+0000", 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2162999999999997e-05 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459228+0000", 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9707000000000001e-05 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.459307+0000", 2020-11-16T22:32:27.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027542809999999999 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462062+0000", 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4033000000000002e-05 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.462126+0000", 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024684660000000001 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:36.486810+0000", 2020-11-16T22:32:27.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.027666455999999999 2020-11-16T22:32:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:27.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.334 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:27.334 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 142.38226315399999, 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:32:27.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:32:27.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:32:27.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:27.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:32:27.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:32:27.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:32:27.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:32:27.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.545 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:32:27.546 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:27.547 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:27.547 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:27.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.547 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:27.750 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 142.083886095, 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:27.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:32:27.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:27.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:32:27.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:27.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:32:27.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:32:27.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:32:27.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:32:27.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:27.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:27.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:32:27.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:32:28.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:32:28.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:28.168 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 81.206938081000004, 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:28.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:32:28.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:28.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:32:28.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:32:28.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:28.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:32:28.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:28.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:32:28.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:32:28.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:28.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:32:28.379 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.582 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:28.582 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.582 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.154340094999995, 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:28.583 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.584 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:32:28.585 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.586 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:28.587 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:32:28.587 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:32:28.587 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.587 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:28.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:32:28.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:32:28.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:28.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:32:28.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:28.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.153296768999994, 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:28.795 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:28.795 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:28.998 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:28.998 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:32:28.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.998 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:28.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:32:29.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:32:29.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:32:29.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:29.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:32:29.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:29.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:29.209 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:29.209 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:32:29.210 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:29.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:32:29.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:32:29.211 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2020-11-16T22:32:29.211 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2020-11-16T22:32:29.213 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x63e4f5a 2020-11-16T22:32:29.213 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:29.214 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:29.224+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:29.310 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:29.325+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:29.361 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:39077 3.6 3:715f5c78:::200.00000013:head [write 3672811~1923 [fadvise_dontneed] in=1923b] snapc 0=[] ondisk+write+known_if_redirected+full_force e167)", 2020-11-16T22:32:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:29.372859+0000", 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0033217279999999999, 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003340197, 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372859+0000", 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372859+0000", 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5661000000000003e-05 2020-11-16T22:32:29.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372894+0000", 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0329999999999997e-06 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372900+0000", 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1106e-05 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372911+0000", 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1470000000000003e-06 2020-11-16T22:32:29.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372916+0000", 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1081999999999998e-05 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.372947+0000", 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5315999999999996e-05 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.373012+0000", 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115485 2020-11-16T22:32:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.373127+0000", 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020665 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:29.375148+0000", 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022894949999999999 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:32:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:32:29.370 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:29.372 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:29.381 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:29.403 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:29.404 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:29.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:29.406 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:29.412 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:29.427+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:29.511 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:29.526+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:29.566 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:29.566 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:29.578 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:29.612 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:29.627+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:29.713 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:29.727+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:29.761 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:29.761 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:29.771 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:29.815 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:29.830+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:29.915 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:29.915 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:29.916 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:29.928+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:29.926 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.0 primary-affinity to 0 (802) 2020-11-16T22:32:29.956 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:32:30.014 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:30.029+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:30.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~72s 2020-11-16T22:32:30.109 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.110 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.115 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:30.130+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:30.119 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:30.216 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:30.230+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:30.307 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.308 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.317 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:30.331+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:30.318 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:30.418 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:30.432+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.485 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:30.519 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:30.533+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:30.619 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:30.634+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:30.635 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.635 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.646 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:30.719 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:30.734+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:30.796 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.797 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.807 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:30.848 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:30.863+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:30.922 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:30.936+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:30.964 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:30.964 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:30.974 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:31.022 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:31.036+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:31.123 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:31.137+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:31.131 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:31.131 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:31.143 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:31.226 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:31.241+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:31.324 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:31.339+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:31.340 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:31.341 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:31.354 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:31.407 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:31.426 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:31.440+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:31.500 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:31.500 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:31.511 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:31.526 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:31.541+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:31.556 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:31.579 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:31.579 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:31.580 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:31.580 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:31.580 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:31.582 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:31.582 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:31.627 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:31.642+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:31.713 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:31.713 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:31.714 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:32:31.714 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:32:31.714 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:31.724 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:31.767 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:31.782+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:31.829 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:31.843+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:31.879 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:32:31.880 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 153.89283960700001, 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:31.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:32:31.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:32:31.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:32:31.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:31.889 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 152.68468944, 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:31.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:32:31.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:32:31.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:32:31.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:32:31.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:32:31.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:32:31.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:32.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:32:32.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 121.60397358500001, 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:32.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:32.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:32.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:32.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:32.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:32.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:32:32.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:32.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:32:32.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:32:32.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:32.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:32:32.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:32:32.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:32.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:32:32.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:32.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:32.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 121.603878145, 2020-11-16T22:32:32.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:32:32.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:32.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:32.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:32.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:32.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:32:32.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:32:32.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:32.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:32:32.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:32:32.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:32:32.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:32.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:32:32.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:32.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:32.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:32.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:32:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:32:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 121.601343813, 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:32:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:32:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:32:33.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:32:33.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:33.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:32:33.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:32:33.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:33.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:32:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:33.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:32:33.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:32:33.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:33.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 121.25092402, 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:32:33.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:33.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:32:33.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:32:33.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:33.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:32:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:32:33.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:33.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:32:33.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:32:33.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:33.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:32:33.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.983 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30480 4.5 4:a611fa68:::1000000188a.00000002:head [write 647168~38050 [266@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.88123813, 2020-11-16T22:32:33.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998, 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30480 2020-11-16T22:32:33.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:33.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:33.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:33.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011669+0000", 2020-11-16T22:32:33.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5147e-05 2020-11-16T22:32:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011714+0000", 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.921e-06 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011719+0000", 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0803e-05 2020-11-16T22:32:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011730+0000", 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6590000000000004e-06 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011736+0000", 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.887e-05 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011765+0000", 2020-11-16T22:32:34.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1990000000000002e-05 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011817+0000", 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9747999999999995e-05 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:34.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.011906+0000", 2020-11-16T22:32:34.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032378683999999998 2020-11-16T22:32:34.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:34.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044285+0000", 2020-11-16T22:32:34.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00062116299999999999 2020-11-16T22:32:34.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044906+0000", 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8942999999999994e-05 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044975+0000", 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6403999999999998e-05 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.045022+0000", 2020-11-16T22:32:34.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.033352331999999998 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18688 3.0 3:1e8917d7:::200.0000000e:head [write 1955503~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:34.401 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.880503907, 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998, 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:34.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:34.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:34.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:34.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012404+0000", 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067999999999997e-05 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012439+0000", 2020-11-16T22:32:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4060000000000004e-06 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012444+0000", 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2590000000000006e-06 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012453+0000", 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9699999999999998e-06 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.012458+0000", 2020-11-16T22:32:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.031711477000000002 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044170+0000", 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799000000000002e-05 2020-11-16T22:32:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044225+0000", 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:34.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.044332+0000", 2020-11-16T22:32:34.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031228390000000001 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.047455+0000", 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011196590000000001 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048574+0000", 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6539000000000003e-05 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:34.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048621+0000", 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9144000000000001e-05 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:31.048650+0000", 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036246346999999998 2020-11-16T22:32:34.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30854 4.5 4:a611fa68:::1000000188a.00000002:head [write 1947041~51687 [295@0]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:34.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 119.818624668, 2020-11-16T22:32:34.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999, 2020-11-16T22:32:34.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:34.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:34.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:35.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:35.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:35.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30854 2020-11-16T22:32:35.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.022 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074283+0000", 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844999999999997e-05 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074325+0000", 2020-11-16T22:32:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.023e-06 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074330+0000", 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3849000000000002e-05 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074354+0000", 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1710000000000004e-06 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074358+0000", 2020-11-16T22:32:35.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5124999999999998e-05 2020-11-16T22:32:35.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074413+0000", 2020-11-16T22:32:35.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3758999999999998e-05 2020-11-16T22:32:35.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074467+0000", 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9667000000000006e-05 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.074566+0000", 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001356601 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.075923+0000", 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027965759999999999 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103889+0000", 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2929999999999997e-05 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103932+0000", 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4254e-05 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:32.103976+0000", 2020-11-16T22:32:35.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029692983999999999 2020-11-16T22:32:35.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.235 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.438 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:35.438 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:35.438 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 118.548529183, 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:35.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:32:35.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:32:35.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:32:35.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:32:35.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:32:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:32:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.808018302999997, 2020-11-16T22:32:35.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:35.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:32:35.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:32:35.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.662 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.349071844999997, 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:32:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:32:35.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:32:35.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:35.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.667000864000002, 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:32:35.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:32:35.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:32:35.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:32:35.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:35.684 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.666554990999998, 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:35.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:35.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:32:35.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.755357931999995, 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:32:35.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:32:35.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:32:35.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.703 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.974900112, 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:35.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:35.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:32:35.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:32:35.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:32:35.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:32:35.713 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.868261191999999, 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:35.714 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:32:35.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:32:35.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:32:35.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:32:35.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.725 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:54217 4.5 4:a611fa68:::1000000188a.00000002:head [read 937984~24576 [2500@0]] snapc 0=[] ondisk+read+known_if_redirected e159)", 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.334519743999998, 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999, 2020-11-16T22:32:35.726 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 54217 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.727 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.558388+0000", 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023326738 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581714+0000", 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.36e-05 2020-11-16T22:32:35.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581728+0000", 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3904999999999999e-05 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581752+0000", 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1073999999999999e-05 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581763+0000", 2020-11-16T22:32:35.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4217999999999998e-05 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581857+0000", 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013256300000000001 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.581990+0000", 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1364000000000002e-05 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:57.582071+0000", 2020-11-16T22:32:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023683461999999999 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:32400 3.0 3:1cc859d9:::200.00000012:head [write 2817827~1931 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e159)", 2020-11-16T22:32:35.734 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.711711928, 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999, 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181196+0000", 2020-11-16T22:32:35.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4440999999999998e-05 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181230+0000", 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4779999999999999e-06 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181235+0000", 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2499999999999995e-06 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181245+0000", 2020-11-16T22:32:35.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2200000000000003e-06 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181249+0000", 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6591000000000003e-05 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181286+0000", 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3828e-05 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181339+0000", 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010155800000000001 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.181441+0000", 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023834762999999998 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205276+0000", 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000670226 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205946+0000", 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4202999999999997e-05 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.205990+0000", 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6330999999999999e-05 2020-11-16T22:32:35.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:58.206016+0000", 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024820888999999999 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:39882 4.5 4:b0fbeeaa:::1000000188d.00000016:head [trimtrunc 349@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:31.747187+0000", 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.14572032700000001, 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023548085, 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:35.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747187+0000", 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747187+0000", 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3992000000000002e-05 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:35.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747231+0000", 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747237+0000", 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.26e-05 2020-11-16T22:32:35.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747250+0000", 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860000000000003e-06 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.747255+0000", 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017521554000000002 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.764776+0000", 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3513e-05 2020-11-16T22:32:35.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.764840+0000", 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000151917 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.764992+0000", 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032917290000000002 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.768284+0000", 2020-11-16T22:32:35.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023735789999999998 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.770657+0000", 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8467999999999997e-05 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.770706+0000", 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9587e-05 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:31.770735+0000", 2020-11-16T22:32:35.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023548085 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:35.755 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:35.756 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:32:35.757 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:35.757 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:32:35.757 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:32:35.758 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:32:35.758 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2020-11-16T22:32:35.759 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x63e65d6 2020-11-16T22:32:35.760 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:35.771+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:35.760 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:35.857 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:35.871+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:35.902 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:35.903 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:35.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:41291 4.5 e168/117 4:b0fbeeaa:::1000000188d.00000016:head v 168'6411, mlcod=168'6405)", 2020-11-16T22:32:35.903 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.911661+0000", 2020-11-16T22:32:35.903 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0062336969999999998, 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0062581069999999997, 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:35.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911661+0000", 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911661+0000", 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1687999999999996e-05 2020-11-16T22:32:35.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911723+0000", 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6380000000000004e-06 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911731+0000", 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1861e-05 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911742+0000", 2020-11-16T22:32:35.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3719999999999998e-06 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.911747+0000", 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001514198 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.913261+0000", 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2894000000000001e-05 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.913274+0000", 2020-11-16T22:32:35.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0016126510000000001 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:32:35.910 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:35.914 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:35.927 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:35.948 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:35.949 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:35.949 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:35.949 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:35.951 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:35.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:35.998 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:36.013+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:36.060 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.074+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.088 INFO:tasks.ceph.mgr.x.smithi138.stderr:2020-11-16T22:32:36.101+0000 7f410bec3700 -1 mgr.server reply reply (1) Operation not permitted no change 2020-11-16T22:32:36.088 INFO:teuthology.orchestra.run.smithi138.stdout:moved 4 / 50 (8%) 2020-11-16T22:32:36.088 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 9 -> 10.4403 (expected baseline 3.53553) 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 16 -> 12 pgs (0.64 -> 0.48 * mean) 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 16 -> 12 pgs (0.64 -> 0.48 * mean) 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:32:36.089 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:36.090 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0022 2020-11-16T22:32:36.090 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0024 2020-11-16T22:32:36.090 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 0.9000 -> 0.6460 2020-11-16T22:32:36.090 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:32:36.093 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:36.093 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:36.101 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:36.102 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2020-11-16T22:32:36.103 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:36.160 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.175+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.261 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.276+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.290 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:36.290 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:36.301 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:36.362 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.376+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.461 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:36.462 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:36.464 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.479+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.473 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:36.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~78s 2020-11-16T22:32:36.564 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.580+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.658 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:36.658 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:36.665 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.681+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.669 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:36.766 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:36.781+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:36.851 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:36.852 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:36.862 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:36.905 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:36.920+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:36.968 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:36.982+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:36.984 INFO:teuthology.orchestra.run.smithi138.stdout:moved 4 / 50 (8%) 2020-11-16T22:32:36.984 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:36.984 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 9 -> 10.4403 (expected baseline 3.53553) 2020-11-16T22:32:36.984 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 16 -> 12 pgs (0.64 -> 0.48 * mean) 2020-11-16T22:32:36.984 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.1 with 16 -> 12 pgs (0.64 -> 0.48 * mean) 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0022 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0024 2020-11-16T22:32:36.985 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 0.9000 -> 0.6460 2020-11-16T22:32:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:32:36.999 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:36.999 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2020-11-16T22:32:37.017 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.017 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.027 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:37.068 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:37.083+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:37.170 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:37.184+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:37.193 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.194 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.203 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:37.286 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:37.301+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:37.367 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.368 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.372 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:37.387+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:37.377 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:37.473 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:37.488+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:37.510 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.510 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.519 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:37.573 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:37.588+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:37.674 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:37.689+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:37.684 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.685 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.694 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:37.777 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:37.793+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:37.858 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:37.859 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:37.868 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:37.908 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:37.924+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:37.952 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:37.975 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:37.990+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:37.994 INFO:teuthology.orchestra.run.smithi138.stdout:moved 4 / 50 (8%) 2020-11-16T22:32:37.994 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:37.994 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 10.4403 -> 12.53 (expected baseline 3.53553) 2020-11-16T22:32:37.994 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 12 -> 8 pgs (0.48 -> 0.32 * mean) 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.2 with 18 -> 20 pgs (0.72 -> 0.8 * mean) 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0022 2020-11-16T22:32:37.995 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0024 2020-11-16T22:32:37.996 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 0.6460 -> 0.4636 2020-11-16T22:32:38.008 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-11-16T22:32:38.008 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2020-11-16T22:32:38.009 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:38.009 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:38.019 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:38.076 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:38.092+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:38.104 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:38.124 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:38.124 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:38.124 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:38.125 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:38.125 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:38.127 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:38.127 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:38.177 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:38.193+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:38.185 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:38.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:32:38.186 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:32:38.186 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:32:38.186 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:38.195 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:38.278 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:38.293+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:38.321 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:38.322 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 273.014462632, 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:38.323 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:38.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:38.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:32:38.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:32:38.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:38.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.332 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 268.01395974399998, 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:38.333 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:38.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:32:38.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:32:38.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:38.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:38.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.342 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 261.36800204100001, 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:38.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:38.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:32:38.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:38.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:32:38.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:32:38.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:38.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:32:38.352 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.353 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:38.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.353 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 259.12245496100002, 2020-11-16T22:32:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:32:38.554 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:38.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:38.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:32:38.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:32:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:32:38.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:32:38.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:32:38.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:38.767 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 259.12236157500001, 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:38.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:32:38.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:38.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:32:38.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:32:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:32:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:32:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:32:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:32:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:32:39.182 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:39.183 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 259.12224117900001, 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:39.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:32:39.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:39.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:32:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:32:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:39.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:32:39.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:32:39.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:32:39.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 259.12214111200001, 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:39.397 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:39.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:32:39.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:39.602 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:32:39.602 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:32:39.602 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.602 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:39.603 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:32:39.604 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:39.605 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:32:39.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:39.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:32:39.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:32:39.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 217.72186812800001, 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:39.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:39.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:39.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:32:39.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:39.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:39.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:32:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:32:40.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:32:40.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:32:40.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:32:40.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:40.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:32:40.226 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:32:40.226 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.226 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:40.227 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:32:40.228 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 159.425093054, 2020-11-16T22:32:40.229 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:40.230 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.231 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:32:40.232 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.233 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:32:40.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:32:40.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:32:40.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.440 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 159.12671599500001, 2020-11-16T22:32:40.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:40.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:40.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:40.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:40.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:32:40.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:40.647 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:32:40.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:40.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:32:40.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:32:40.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:40.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:32:40.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:32:40.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:40.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:40.859 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:32:40.859 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:41.062 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 98.249767981000005, 2020-11-16T22:32:41.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:32:41.062 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:41.062 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:41.062 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:41.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:32:41.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:41.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:32:41.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:41.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:32:41.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 93.197169994999996, 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:41.070 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:41.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:41.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:32:41.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:32:41.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:32:41.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:41.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:32:41.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:32:41.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:41.483 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 93.196126668999995, 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:41.484 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:41.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:32:41.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:41.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:32:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:32:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:41.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:32:41.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:41.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:32:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:41.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40505 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6626057669999996, 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001, 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:41.695 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3241e-05 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672457+0000", 2020-11-16T22:32:41.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6820000000000002e-06 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672461+0000", 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9270000000000005e-06 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672469+0000", 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9280000000000003e-06 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672473+0000", 2020-11-16T22:32:41.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7399e-05 2020-11-16T22:32:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672500+0000", 2020-11-16T22:32:41.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1418000000000001e-05 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672552+0000", 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1761000000000006e-05 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672624+0000", 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038091973000000001 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710716+0000", 2020-11-16T22:32:41.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00027118299999999998 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710987+0000", 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5528999999999997e-05 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711032+0000", 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1307999999999999e-05 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711064+0000", 2020-11-16T22:32:41.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001 2020-11-16T22:32:42.110 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.110 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:42.110 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6625303679999996, 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:42.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:42.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:42.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:32:42.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:32:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:32:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:32:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:42.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:32:42.527 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6619633399999998, 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:32:42.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:32:42.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:32:42.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.536 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62890 4.6 4:6090c0e3:::1000000188d.00000003:head [write 1632508~30811 [387@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:42.536 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:42.738 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6614291210000003, 2020-11-16T22:32:42.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997, 2020-11-16T22:32:42.738 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:42.738 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62890 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:42.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:42.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5053000000000003e-05 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673665+0000", 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9869999999999999e-06 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673670+0000", 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0429999999999996e-06 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673679+0000", 2020-11-16T22:32:42.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2259999999999999e-06 2020-11-16T22:32:42.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:42.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673683+0000", 2020-11-16T22:32:42.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6919999999999998e-05 2020-11-16T22:32:42.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673730+0000", 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000105 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673835+0000", 2020-11-16T22:32:42.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119379 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673955+0000", 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033775507000000003 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707730+0000", 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055870140000000004 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713317+0000", 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2094999999999993e-05 2020-11-16T22:32:42.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713389+0000", 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4565000000000001e-05 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:42.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713474+0000", 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62892 4.3 4:db4b432d:::1000000188d.00000005:head [write 2785386~48952 [387@2688279]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6613127219999999, 2020-11-16T22:32:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001, 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62892 2020-11-16T22:32:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4876000000000006e-05 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673801+0000", 2020-11-16T22:32:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2430000000000002e-06 2020-11-16T22:32:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673807+0000", 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.226e-05 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673820+0000", 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1309999999999997e-06 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673825+0000", 2020-11-16T22:32:43.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3609000000000003e-05 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673858+0000", 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0615999999999997e-05 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673919+0000", 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010215100000000001 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674021+0000", 2020-11-16T22:32:43.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033782754999999998 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707804+0000", 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055369219999999997 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713341+0000", 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3466000000000005e-05 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713404+0000", 2020-11-16T22:32:43.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8748999999999998e-05 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713443+0000", 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.368 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:62889 4.2 e168/163 4:5e663dd9:::1000000188d.00000002:head v 168'1276, mlcod=168'1274)", 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4.6608979020000003, 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998, 2020-11-16T22:32:43.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:43.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4456999999999995e-05 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:43.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674216+0000", 2020-11-16T22:32:43.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7149999999999998e-06 2020-11-16T22:32:43.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674223+0000", 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0988999999999999e-05 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674234+0000", 2020-11-16T22:32:43.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9389999999999996e-06 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674239+0000", 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0126999999999997e-05 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674279+0000", 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6388999999999998e-05 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674296+0000", 2020-11-16T22:32:43.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038984000999999997 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713280+0000", 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1035000000000002e-05 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713331+0000", 2020-11-16T22:32:43.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998 2020-11-16T22:32:43.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.782 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:43.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 2.5379330019999999, 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:43.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:43.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:43.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:32:43.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:32:43.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:32:43.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:32:43.993 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.994 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:43.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.994 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:43.994 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:43.994 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:43.995 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:32:43.996 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:43.998 INFO:tasks.ceph.mgr.x.smithi138.stderr:2020-11-16T22:32:38.357+0000 7f410bec3700 -1 mgr.server reply reply (1) Operation not permitted no change 2020-11-16T22:32:43.998 INFO:teuthology.orchestra.run.smithi138.stdout:moved 2 / 50 (4%) 2020-11-16T22:32:43.998 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:43.998 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 12.53 -> 13.7295 (expected baseline 3.53553) 2020-11-16T22:32:43.998 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 8 -> 6 pgs (0.32 -> 0.24 * mean) 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.2 with 20 -> 21 pgs (0.8 -> 0.84 * mean) 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:oload 110 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 0.05 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0022 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0024 2020-11-16T22:32:43.999 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 weight 0.4636 -> 0.4136 2020-11-16T22:32:44.000 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:32:44.000 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2020-11-16T22:32:44.000 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2020-11-16T22:32:44.001 INFO:tasks.workunit.client.0.smithi138.stdout:truncating to largest ever: 0x63f1359 2020-11-16T22:32:44.001 INFO:tasks.workunit.client.0.smithi138.stdout:fallocating to largest ever: 0x6400000 2020-11-16T22:32:44.001 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:44.010+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:44.002 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:44.099 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:44.113+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:44.161 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:44.161 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:43046 3.4 3:3939b497:::200.00000014:head [write 1577587~1923 [fadvise_dontneed] in=1923b] snapc 0=[] ondisk+write+known_if_redirected+full_force e172)", 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:44.171606+0000", 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.004869338, 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048879470000000001, 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:44.162 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171606+0000", 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171606+0000", 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1292999999999997e-05 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171677+0000", 2020-11-16T22:32:44.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4129999999999995e-06 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171686+0000", 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5931999999999999e-05 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171702+0000", 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5169999999999996e-06 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171709+0000", 2020-11-16T22:32:44.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5800999999999999e-05 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171735+0000", 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7426000000000001e-05 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:44.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171802+0000", 2020-11-16T22:32:44.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114401 2020-11-16T22:32:44.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:44.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.171916+0000", 2020-11-16T22:32:44.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035878889999999999 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175504+0000", 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0038986720000000002 2020-11-16T22:32:44.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:43047 3.4 3:3939b497:::200.00000014:head [write 1579510~1927 [fadvise_dontneed] in=1927b] snapc 0=[] ondisk+write+known_if_redirected+full_force e172)", 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:44.175673+0000", 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00080228000000000005, 2020-11-16T22:32:44.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000873849, 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175673+0000", 2020-11-16T22:32:44.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:44.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:44.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175673+0000", 2020-11-16T22:32:44.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3009999999999994e-05 2020-11-16T22:32:44.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:44.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175746+0000", 2020-11-16T22:32:44.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1190000000000003e-06 2020-11-16T22:32:44.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175755+0000", 2020-11-16T22:32:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6320999999999998e-05 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175771+0000", 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8399999999999997e-06 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175778+0000", 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9788e-05 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175808+0000", 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6165000000000001e-05 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175874+0000", 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100113 2020-11-16T22:32:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:44.175974+0000", 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030135600000000002 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:32:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:32:44.583 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:44.585 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:44.586 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:44.586 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:44.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:44.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:44.790 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:44.791 INFO:tasks.workunit.client.0.smithi138.stdout:10000 punch from 0xc44d4b to 0xc4b890, (0x6b45 bytes) 2020-11-16T22:32:44.791 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:44.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:moved 0 / 50 (0%) 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:avg 25 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:stddev 12.53 -> 12.53 (expected baseline 3.53553) 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:min osd.1 with 8 -> 8 pgs (0.32 -> 0.32 * mean) 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:max osd.2 with 20 -> 20 pgs (0.8 -> 0.8 * mean) 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:44.792 INFO:teuthology.orchestra.run.smithi138.stdout:oload 1000 2020-11-16T22:32:44.793 INFO:teuthology.orchestra.run.smithi138.stdout:max_change 1 2020-11-16T22:32:44.793 INFO:teuthology.orchestra.run.smithi138.stdout:max_change_osds 4 2020-11-16T22:32:44.793 INFO:teuthology.orchestra.run.smithi138.stdout:average_utilization 0.0022 2020-11-16T22:32:44.793 INFO:teuthology.orchestra.run.smithi138.stdout:overload_utilization 0.0223 2020-11-16T22:32:44.793 INFO:teuthology.orchestra.run.smithi138.stderr:no change 2020-11-16T22:32:44.794 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:44.794 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:44.598+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:44.795 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:44.810+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:44.896 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:44.911+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:44.944 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:44.944 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:44.954 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:44.997 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.012+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:32:45.098 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.113+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.116 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.116 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:45.128 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:45.199 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.214+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.271 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.271 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:45.282 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:45.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~86s 2020-11-16T22:32:45.326 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:45.341+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:45.401 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.416+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.422 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.422 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:45.433 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:45.502 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.517+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.603 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:45.618+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:45.625 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.626 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:45.638 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:45.704 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:45.718+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:45.796 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.796 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:45.805 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:45.820+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:45.807 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:45.906 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:45.921+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:45.996 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:45.996 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.006 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:46.007 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.022+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.109 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:46.124+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:46.162 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:46.163 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.174 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:46.217 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:46.232+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:46.310 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.324+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.323 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:46.324 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.334 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:46.417 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:46.432+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:46.511 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:46.526+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:46.516 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:46.516 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.527 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:46.613 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.627+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.708 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:46.708 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.713 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.727+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.718 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:46.792 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:46.813 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.828+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.867 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:46.867 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:46.879 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:46.920 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:46.935+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:46.942 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:46.963 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:46.966 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:46.966 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:47.014 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:47.029+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:47.036 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:32:47.062 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:47.062 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:32:47.063 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:32:47.063 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:32:47.063 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:47.072 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:47.115 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:47.130+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:47.216 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:47.230+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:47.229 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:47.229 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:32:47.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 169.242275681, 2020-11-16T22:32:47.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:47.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:32:47.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:32:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:32:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 168.03412551400001, 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:32:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:32:47.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:32:47.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:32:47.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:32:47.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:47.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:32:47.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:47.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:32:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:32:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:47.455 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 136.95340965899999, 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:47.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:47.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:32:47.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:32:47.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:47.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:32:47.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:32:47.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:32:47.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:47.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.669 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 136.95331421899999, 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:47.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:32:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:32:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:32:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:48.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:32:48.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:32:48.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:48.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 136.95077988700001, 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:32:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:48.087 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:48.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:48.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:48.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:32:48.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:48.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:32:48.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:32:48.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.299 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:48.300 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 136.600360094, 2020-11-16T22:32:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:32:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:32:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:32:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:48.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:32:48.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:32:48.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:48.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:32:48.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:48.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:32:48.714 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.715 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:48.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:48.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.918 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:32:48.918 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 133.89796525700001, 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:32:48.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:32:48.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:32:48.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:48.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:48.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:49.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:32:49.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:32:49.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:32:49.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:32:49.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:32:49.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:49.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.15745437699999, 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:49.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:32:49.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:49.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:32:49.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:32:49.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:49.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:32:49.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:32:49.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:49.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:32:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:32:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 104.69850791899999, 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:32:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:32:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:32:49.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:32:49.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:49.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:49.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:32:49.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:32:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.964 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.016436938, 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:49.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:32:49.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:32:49.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:32:49.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:32:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:50.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:32:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.01599106499999, 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:50.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:32:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.179 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:50.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:50.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:32:50.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:32:50.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:50.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:32:50.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:32:50.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:50.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:32:50.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.10479400600001, 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:32:50.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:32:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:50.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:32:50.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:50.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:50.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:32:50.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:32:50.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:50.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:32:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.324336185999996, 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:50.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:50.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:50.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:50.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:50.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:51.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:32:51.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:32:51.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:32:51.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:32:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:51.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:32:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:32:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:32:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.217697266000002, 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:32:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:32:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:32:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:32:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:51.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:51.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.436 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:40504 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 389@2510211] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:51.638 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:32:51.638 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.569796717999999, 2020-11-16T22:32:51.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998, 2020-11-16T22:32:51.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:51.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3415000000000003e-05 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:51.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672600+0000", 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8559999999999996e-06 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672609+0000", 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5803000000000001e-05 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672625+0000", 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0939999999999996e-06 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:51.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672631+0000", 2020-11-16T22:32:51.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4663999999999998e-05 2020-11-16T22:32:51.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:51.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672676+0000", 2020-11-16T22:32:51.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111723 2020-11-16T22:32:51.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672787+0000", 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015558 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672943+0000", 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063655059999999999 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.679308+0000", 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.034442592000000001 2020-11-16T22:32:51.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713751+0000", 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7538000000000002e-05 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:51.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713829+0000", 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2500999999999998e-05 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713881+0000", 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998 2020-11-16T22:32:51.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62889 4.2 4:5e663dd9:::1000000188d.00000002:head [write 2580480~40960 [389@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.568742615, 2020-11-16T22:32:51.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477, 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62889 2020-11-16T22:32:51.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7012000000000001e-05 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673678+0000", 2020-11-16T22:32:51.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.464e-06 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673685+0000", 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2585999999999999e-05 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:51.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673698+0000", 2020-11-16T22:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1329999999999998e-06 2020-11-16T22:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673703+0000", 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180645 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673884+0000", 2020-11-16T22:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1970999999999993e-05 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673946+0000", 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103269 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674049+0000", 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035807249999999999 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.677630+0000", 2020-11-16T22:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036176901999999997 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713807+0000", 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1492e-05 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:52.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713868+0000", 2020-11-16T22:32:52.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129278 2020-11-16T22:32:52.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713997+0000", 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62891 4.5 4:ab64ebfc:::1000000188d.00000009:head [write 132310~6636 [388@0]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.568595928000001, 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002, 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:52.268 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62891 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:52.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7964999999999998e-05 2020-11-16T22:32:52.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673806+0000", 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5039999999999999e-06 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673812+0000", 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2136999999999999e-05 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673824+0000", 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2020-11-16T22:32:52.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673829+0000", 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018687 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:52.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675698+0000", 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9197999999999997e-05 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675767+0000", 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114926 2020-11-16T22:32:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675882+0000", 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058666569999999999 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:52.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.681749+0000", 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032181653999999997 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713930+0000", 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7389000000000003e-05 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713978+0000", 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.236e-05 2020-11-16T22:32:52.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714030+0000", 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:52.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62893 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2777088~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.567936207000001, 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001, 2020-11-16T22:32:52.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62893 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:52.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:52.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:32:52.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7876999999999997e-05 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674445+0000", 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674452+0000", 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4151999999999999e-05 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674467+0000", 2020-11-16T22:32:52.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0669999999999997e-06 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674473+0000", 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066222999999999998 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675135+0000", 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100586 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675236+0000", 2020-11-16T22:32:52.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038787833000000001 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714023+0000", 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1931e-05 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714085+0000", 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013896399999999999 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714224+0000", 2020-11-16T22:32:52.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62894 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2854912~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:32:52.693 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.567823478999999, 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997, 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:52.694 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62894 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:32:52.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5401000000000002e-05 2020-11-16T22:32:52.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674555+0000", 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6089999999999997e-06 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674562+0000", 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0671e-05 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674573+0000", 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2869999999999998e-06 2020-11-16T22:32:52.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674577+0000", 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068503599999999998 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675262+0000", 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0427000000000003e-05 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675323+0000", 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038926022999999997 2020-11-16T22:32:52.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714249+0000", 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3566999999999997e-05 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714302+0000", 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134485 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714437+0000", 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:32:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.2782651640000005, 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:32:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:32:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:32:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:53.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:32:53.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:32:53.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:32:53.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:53.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:53.322 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:53.323 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:53.324 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:32:53.526 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:32:53.527 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:32:53.528 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:32:53.529 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:53.338+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:53.530 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:53.569 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:53.584+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:53.632 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:53.646+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:53.685 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:32:53.686 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:32:53.686 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:53.696 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:53.732 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:53.747+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:53.745 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:53.766 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:53.766 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:53.766 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:53.767 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:53.767 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:53.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:53.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:53.833 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:53.847+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:53.888 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:53.888 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:53.900 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 174, 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:32:53.901 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:32:48.038720+0000", 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:31:03.095061+0000", 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5801984, 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "noscrub", 2020-11-16T22:32:53.902 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:32:53.903 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "luminous", 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:32:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:32:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:32:53.906 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:32:53.907 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.908 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:32:53.909 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:32:53.910 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:32:53.911 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:32:53.912 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:32:53.913 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:32:53.915 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:32:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:32:53.917 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:32:53.918 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:32:53.919 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:32:53.920 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:32:53.921 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:32:53.922 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:32:53.923 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:32:53.923 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:32:53.923 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:32:53.923 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:32:53.924 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:32:53.925 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:32:53.926 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:32:53.927 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:32:53.928 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:32:53.929 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:32:53.930 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:32:53.931 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:32:53.932 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:32:53.933 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:32:53.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.935 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:32:53.936 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:32:53.937 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:32:53.938 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:32:53.939 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:32:53.940 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:32:53.941 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:32:53.942 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:32:53.943 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:32:53.944 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 0, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:32:53.946 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 119, 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 141, 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.947 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:32:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:32:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.954 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: "exists" 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 0.463592529296875, 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0.7154998779296875, 2020-11-16T22:32:53.956 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 170, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.958 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.959 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:32:53.960 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.962 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:32:53.963 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:32:53.965 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:32:53.966 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 63, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 69, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 108, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 171, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 70, 2020-11-16T22:32:53.967 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.969 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:32:53.971 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.972 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:32:53.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:32:53.974 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/2959", 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/2959", 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/2959", 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/2959", 2020-11-16T22:32:53.975 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:32:53.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:32:53.977 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 141 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:32:53.978 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:32:53.979 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 70 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.980 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [ 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.7", 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: 0 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.981 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [ 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "1.0", 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.982 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.0", 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.983 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.3", 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.984 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.5", 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.6", 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.7", 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.2", 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.4", 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.6", 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.0", 2020-11-16T22:32:53.990 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.1", 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.991 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.2", 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.992 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.4", 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:32:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.6", 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2020-11-16T22:32:53.994 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:32:53.995 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:32:53.996 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:32:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:32:53.998 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:32:53.998 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:32:53.998 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:32:53.998 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:32:53.999 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.013+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.000 INFO:tasks.thrashosds.thrasher:j is {'epoch': 174, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:32:48.038720+0000', 'last_up_change': '2020-11-16T22:31:03.095061+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 119, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 0.463592529296875, 'primary_affinity': 0.7154998779296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 170, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 63, 'last_clean_end': 69, 'up_from': 108, 'up_thru': 171, 'down_at': 70, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 2959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 2959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 2959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 2959}]}, 'public_addr': '172.21.15.122:6801/2959', 'cluster_addr': '172.21.15.122:6803/2959', 'heartbeat_back_addr': '172.21.15.122:6807/2959', 'heartbeat_front_addr': '172.21.15.122:6805/2959', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 70}], 'pg_upmap': [{'pgid': '4.7', 'osds': [2, 0]}], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '1.0', 'osds': [2, 1]}, {'pgid': '2.0', 'osds': [2, 1]}, {'pgid': '2.3', 'osds': [2, 1]}, {'pgid': '2.5', 'osds': [2, 1]}, {'pgid': '2.6', 'osds': [2, 1]}, {'pgid': '2.7', 'osds': [2, 1]}, {'pgid': '3.2', 'osds': [1, 2]}, {'pgid': '3.4', 'osds': [2, 1]}, {'pgid': '3.6', 'osds': [2, 1]}, {'pgid': '4.0', 'osds': [2, 1]}, {'pgid': '4.1', 'osds': [2, 1]}, {'pgid': '4.2', 'osds': [2, 1]}, {'pgid': '4.4', 'osds': [2, 1]}, {'pgid': '4.6', 'osds': [1, 2]}], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:32:54.001 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.7 2020-11-16T22:32:54.001 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 4.7 2020-11-16T22:32:54.100 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.115+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.142 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.142 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.152 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:54.201 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:54.215+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:54.297 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.298 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.302 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:54.316+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:54.310 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:54.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~96s 2020-11-16T22:32:54.403 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:54.416+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:54.462 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.462 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.474 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:54.516 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.531+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.605 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.619+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.626 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.626 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.636 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:54.706 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.720+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.791 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.792 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.803 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:54.806 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.820+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.907 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:54.921+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:54.957 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:54.957 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:54.967 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:55.010 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.024+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.055 INFO:teuthology.orchestra.run.smithi138.stderr:clear 4.7 pg_upmap mapping 2020-11-16T22:32:55.105 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:55.105 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:55.108 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:55.123+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:55.115 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:55.209 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.222+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.268 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:55.268 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:55.278 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:55.310 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.324+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.411 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.424+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.435 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:55.436 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:55.449 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:32:55.511 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.525+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.534 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:32:55.583 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:55.584 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:55.595 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:32:55.638 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.652+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.713 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.726+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.752 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:32:55.753 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:32:55.764 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:32:55.770 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:32:55.848 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.862+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.914 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:55.927+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:32:55.919 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:55.943 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:32:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:32:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:55.945 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:55.958 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:55.958 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:32:55.958 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:32:55.958 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:32:55.958 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:55.970 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:32:56.014 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:56.029+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:32:56.115 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:56.129+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:56.152 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:32:56.152 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:32:56.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 290.845113846, 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:32:56.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:32:56.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:32:56.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:32:56.162 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 285.84461095799998, 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:56.163 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:32:56.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:32:56.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 279.19865325500001, 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:32:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:32:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 276.95310617500002, 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:32:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:32:56.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:32:56.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:32:56.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:32:56.195 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:56.196 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:56.398 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 276.95301278900001, 2020-11-16T22:32:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:32:56.445 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:32:56.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:32:56.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.454 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 276.95289239300001, 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:32:56.455 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:32:56.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:32:56.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:32:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:32:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:32:56.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:32:56.465 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 276.95279232600001, 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:32:56.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:32:56.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:56.475 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 235.55251934200001, 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:56.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:32:56.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:32:56.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:32:56.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:32:56.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:56.485 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 177.255744268, 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:56.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:32:56.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:32:56.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:32:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:32:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 176.95736720900001, 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:32:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:32:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:32:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:32:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 116.080419195, 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:32:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:32:56.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:32:56.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.916 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 111.027821209, 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:32:56.917 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:32:56.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:32:56.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:56.924 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 111.02677788299999, 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:56.925 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:56.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:32:56.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:56.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:56.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:32:57.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:32:57.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:32:57.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.133 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40505 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.493256980999998, 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001, 2020-11-16T22:32:57.134 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3241e-05 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672457+0000", 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6820000000000002e-06 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672461+0000", 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9270000000000005e-06 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672469+0000", 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9280000000000003e-06 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672473+0000", 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7399e-05 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672500+0000", 2020-11-16T22:32:57.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1418000000000001e-05 2020-11-16T22:32:57.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672552+0000", 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1761000000000006e-05 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672624+0000", 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038091973000000001 2020-11-16T22:32:57.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710716+0000", 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00027118299999999998 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:57.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710987+0000", 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5528999999999997e-05 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711032+0000", 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1307999999999999e-05 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711064+0000", 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.493181581999998, 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.347 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:32:57.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:32:57.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.554 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.554 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.554 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:32:57.554 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.555 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:32:57.556 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.557 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:32:57.558 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:57.559 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.492614553999999, 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:57.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:32:57.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:32:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:32:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:32:57.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62890 4.6 4:6090c0e3:::1000000188d.00000003:head [write 1632508~30811 [387@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.492080335000001, 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997, 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.770 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62890 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5053000000000003e-05 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673665+0000", 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9869999999999999e-06 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673670+0000", 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0429999999999996e-06 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673679+0000", 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2259999999999999e-06 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673683+0000", 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6919999999999998e-05 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673730+0000", 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000105 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673835+0000", 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119379 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673955+0000", 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033775507000000003 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707730+0000", 2020-11-16T22:32:57.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055870140000000004 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713317+0000", 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2094999999999993e-05 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713389+0000", 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4565000000000001e-05 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713474+0000", 2020-11-16T22:32:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62892 4.3 4:db4b432d:::1000000188d.00000005:head [write 2785386~48952 [387@2688279]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.491963936000001, 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001, 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62892 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4876000000000006e-05 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673801+0000", 2020-11-16T22:32:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2430000000000002e-06 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673807+0000", 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.226e-05 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:57.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673820+0000", 2020-11-16T22:32:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1309999999999997e-06 2020-11-16T22:32:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673825+0000", 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3609000000000003e-05 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673858+0000", 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0615999999999997e-05 2020-11-16T22:32:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673919+0000", 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010215100000000001 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674021+0000", 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033782754999999998 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707804+0000", 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055369219999999997 2020-11-16T22:32:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713341+0000", 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3466000000000005e-05 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713404+0000", 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8748999999999998e-05 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713443+0000", 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001 2020-11-16T22:32:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:62889 4.2 e168/163 4:5e663dd9:::1000000188d.00000002:head v 168'1276, mlcod=168'1274)", 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.491549116000002, 2020-11-16T22:32:57.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998, 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:57.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4456999999999995e-05 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674216+0000", 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7149999999999998e-06 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674223+0000", 2020-11-16T22:32:57.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0988999999999999e-05 2020-11-16T22:32:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674234+0000", 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9389999999999996e-06 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674239+0000", 2020-11-16T22:32:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0126999999999997e-05 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674279+0000", 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6388999999999998e-05 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674296+0000", 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038984000999999997 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713280+0000", 2020-11-16T22:32:58.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1035000000000002e-05 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713331+0000", 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998 2020-11-16T22:32:58.202 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:58.203 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 20.368584215999999, 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:32:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:32:58.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:32:58.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:32:58.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:32:58.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:32:58.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:32:58.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:32:58.413 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:32:58.414 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:32:58.415 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:32:58.416 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:32:58.417 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:58.429+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:58.515 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:58.529+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:58.571 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:32:58.571 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47316 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 838@1936727] snapc 1=[] ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.576496+0000", 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0088524139999999994, 2020-11-16T22:32:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088776010000000006, 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576496+0000", 2020-11-16T22:32:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576496+0000", 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4430000000000002e-05 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576520+0000", 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5129999999999998e-06 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576524+0000", 2020-11-16T22:32:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8740000000000007e-06 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576534+0000", 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2209999999999999e-06 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576539+0000", 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3361999999999997e-05 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576572+0000", 2020-11-16T22:32:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5936000000000001e-05 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576628+0000", 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5957000000000003e-05 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576724+0000", 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022829299999999999 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47319 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 838@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.576650+0000", 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0086974749999999997, 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008814433, 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576650+0000", 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576650+0000", 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4634000000000002e-05 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576675+0000", 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-06 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576680+0000", 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0060000000000003e-06 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576687+0000", 2020-11-16T22:32:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.695e-06 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576689+0000", 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5778000000000001e-05 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576735+0000", 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012118 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576856+0000", 2020-11-16T22:32:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180256 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577037+0000", 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072983060000000001 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.584335+0000", 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076843650000000003 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47320 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 838@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.576717+0000", 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0086306509999999996, 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088395769999999995, 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576717+0000", 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576717+0000", 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5701e-05 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576743+0000", 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0990000000000001e-06 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576747+0000", 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7599999999999997e-06 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576754+0000", 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.632e-06 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576756+0000", 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037801380000000002 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.580537+0000", 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7621000000000006e-05 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.580634+0000", 2020-11-16T22:32:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000154499 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.580789+0000", 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040714499999999999 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47322 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 838@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.576878+0000", 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0084704189999999999, 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087599519999999997, 2020-11-16T22:32:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:58.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:58.798 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:58.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576878+0000", 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:58.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576878+0000", 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6063000000000001e-05 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576904+0000", 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9809999999999997e-06 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576908+0000", 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4309999999999994e-06 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:58.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576916+0000", 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8679999999999999e-06 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576919+0000", 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075032079999999999 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.584422+0000", 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2844999999999996e-05 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:58.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.584515+0000", 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016055400000000001 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.584675+0000", 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077979499999999997 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47323 4.5 4.e44f3815 (undecoded) ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.576944+0000", 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0084041259999999996, 2020-11-16T22:32:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087831090000000007, 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576944+0000", 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576944+0000", 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3628000000000001e-05 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:58.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576967+0000", 2020-11-16T22:32:58.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8639999999999998e-06 2020-11-16T22:32:59.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576971+0000", 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4670000000000001e-06 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576978+0000", 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4219999999999999e-06 2020-11-16T22:32:59.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.576980+0000", 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6381000000000003e-05 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:47326 4.5 4.5577df0d (undecoded) ondisk+write+known_if_redirected+full_force e176)", 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.577146+0000", 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0082022650000000003, 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086413649999999998, 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:59.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577146+0000", 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577146+0000", 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6378999999999998e-05 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577192+0000", 2020-11-16T22:32:59.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.224e-06 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577198+0000", 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.111e-05 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577209+0000", 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2560000000000004e-06 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.577214+0000", 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7968999999999999e-05 2020-11-16T22:32:59.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:47327 4.3 e176/137 4:cf4ec11c:::1000000188d.00000017:head v 176'12934, mlcod=176'12931)", 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.581307+0000", 2020-11-16T22:32:59.018 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0040405709999999997, 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045348259999999996, 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581307+0000", 2020-11-16T22:32:59.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:59.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:59.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581307+0000", 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6340000000000005e-05 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581374+0000", 2020-11-16T22:32:59.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3219999999999999e-06 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581383+0000", 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2838e-05 2020-11-16T22:32:59.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581396+0000", 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2890000000000003e-06 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.581402+0000", 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001061127 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.582463+0000", 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4127999999999999e-05 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:59.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.582477+0000", 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011700440000000001 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.229 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:68655 4.4 4:218b21fa:::1000000188d.00000001:head [write 2455421~11057 [838@4194304] in=11057b] snapc 1=[] ondisk+write+known_if_redirected e176)", 2020-11-16T22:32:59.229 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.585253+0000", 2020-11-16T22:32:59.229 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.5172000000000003e-05, 2020-11-16T22:32:59.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065865499999999996, 2020-11-16T22:32:59.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 68655 2020-11-16T22:32:59.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.231 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:59.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:59.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585253+0000", 2020-11-16T22:32:59.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585253+0000", 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9838e-05 2020-11-16T22:32:59.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:59.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585303+0000", 2020-11-16T22:32:59.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6910000000000002e-06 2020-11-16T22:32:59.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585308+0000", 2020-11-16T22:32:59.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774000000000001e-05 2020-11-16T22:32:59.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585319+0000", 2020-11-16T22:32:59.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2490000000000003e-06 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585323+0000", 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9178000000000003e-05 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585362+0000", 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106085 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585469+0000", 2020-11-16T22:32:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018534000000000001 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585654+0000", 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00040115499999999999 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:68656 4.4 4.14da82ec (undecoded) ondisk+write+known_if_redirected e176)", 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.585358+0000", 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9999905, 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063960999999999998, 2020-11-16T22:32:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:32:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:32:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:32:59.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:32:59.646 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 68656 2020-11-16T22:32:59.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.646 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585358+0000", 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585358+0000", 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9589000000000001e-05 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585387+0000", 2020-11-16T22:32:59.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.749e-06 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585391+0000", 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2119999999999997e-06 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585398+0000", 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8079999999999998e-06 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585402+0000", 2020-11-16T22:32:59.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4357999999999998e-05 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:68657 4.3 e176/137 4:db4b432d:::1000000188d.00000005:head v 176'12935, mlcod=176'12931)", 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:58.585837+0000", 2020-11-16T22:32:59.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9995112, 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000227531, 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:32:59.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585837+0000", 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585837+0000", 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7772000000000003e-05 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585884+0000", 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1159999999999998e-06 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585890+0000", 2020-11-16T22:32:59.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3240000000000003e-06 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585898+0000", 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8709999999999999e-06 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:32:59.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585902+0000", 2020-11-16T22:32:59.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6149000000000002e-05 2020-11-16T22:32:59.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585938+0000", 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2073999999999999e-05 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:58.585960+0000", 2020-11-16T22:32:59.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000123306 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 10 2020-11-16T22:32:59.860 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:32:59.862 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:32:59.863 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:32:59.863 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:32:59.863 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:32:59.863 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:32:59.863 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:32:59.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:32:59.864 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:32:59.864 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:32:59.865 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:32:59.876+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:32:59.962 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:32:59.976+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:00.062 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.062 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.064 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.078+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.066 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:00.067 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:33:00.067 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:33:00.067 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:33:00.067 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:33:00.068 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:33:00.073 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:00.165 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:00.179+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:00.213 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:00.213 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:33:00.214 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:00.262 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.262 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.266 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.281+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.272 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:00.367 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:00.381+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:00.441 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.442 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.452 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:00.468 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.482+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.569 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.583+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.630 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.630 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.643 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:00.669 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.683+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~102s 2020-11-16T22:33:00.770 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:00.783+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:00.792 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.793 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.804 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:00.870 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:00.885+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:00.951 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:00.951 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:00.962 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:01.003 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.017+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.072 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:01.085+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:01.148 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.149 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.158 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:01.201 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.216+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.273 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:01.286+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:01.306 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.306 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.316 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:01.374 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.388+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.457 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.458 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.474 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:01.475 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:01.488+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:01.576 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:01.589+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:01.614 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.614 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.624 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:01.676 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.690+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.776 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.791+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.811 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.811 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.821 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:01.865 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:01.865 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:33:01.877 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.892+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.978 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:01.992+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:01.980 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:01.980 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:01.990 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:02.019 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:02.040 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:02.040 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:02.040 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:02.040 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:02.040 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:02.043 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:02.043 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:02.079 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:02.092+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:02.179 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:02.193+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:02.195 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:02.195 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:02.196 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:02.196 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:02.196 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:02.206 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:02.280 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:02.293+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:02.381 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:02.395+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:02.393 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:02.393 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 184.406624222, 2020-11-16T22:33:02.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:02.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:02.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:02.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:02.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 183.19847405499999, 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:02.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:02.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:02.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:02.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:02.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 152.1177582, 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 152.11766276, 2020-11-16T22:33:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:02.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:02.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 152.11512842799999, 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:33:02.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:33:02.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:33:02.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:33:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:33:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:02.449 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 151.76470863500001, 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:33:02.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:33:02.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:33:02.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:02.459 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 149.06231379799999, 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:33:02.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:33:02.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:33:02.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:33:02.469 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 122.321802918, 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:33:02.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:33:02.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:33:02.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:02.479 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 119.86285646, 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:02.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:02.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:02.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.489 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 118.18078547899999, 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:33:02.490 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:33:02.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:33:02.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:33:02.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:33:02.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 118.180339606, 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:02.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:33:02.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:33:02.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 117.269142547, 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:33:02.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:33:02.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:33:02.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:33:02.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 105.48868472700001, 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:33:02.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:33:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:33:02.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 91.382045806999997, 2020-11-16T22:33:02.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:02.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:02.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:02.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:40504 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 389@2510211] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.734145259000002, 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998, 2020-11-16T22:33:02.539 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3415000000000003e-05 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672600+0000", 2020-11-16T22:33:02.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8559999999999996e-06 2020-11-16T22:33:02.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672609+0000", 2020-11-16T22:33:02.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5803000000000001e-05 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672625+0000", 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0939999999999996e-06 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672631+0000", 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4663999999999998e-05 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672676+0000", 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111723 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672787+0000", 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015558 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672943+0000", 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063655059999999999 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.679308+0000", 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.034442592000000001 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713751+0000", 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7538000000000002e-05 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713829+0000", 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2500999999999998e-05 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713881+0000", 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62889 4.2 4:5e663dd9:::1000000188d.00000002:head [write 2580480~40960 [389@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:02.753 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.733091156, 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477, 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.754 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62889 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7012000000000001e-05 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673678+0000", 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.464e-06 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673685+0000", 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2585999999999999e-05 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673698+0000", 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1329999999999998e-06 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673703+0000", 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180645 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673884+0000", 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1970999999999993e-05 2020-11-16T22:33:02.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673946+0000", 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103269 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674049+0000", 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035807249999999999 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.677630+0000", 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036176901999999997 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713807+0000", 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1492e-05 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713868+0000", 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129278 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713997+0000", 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.763 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62891 4.5 4:ab64ebfc:::1000000188d.00000009:head [write 132310~6636 [388@0]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.732944469, 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002, 2020-11-16T22:33:02.764 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62891 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.765 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7964999999999998e-05 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673806+0000", 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5039999999999999e-06 2020-11-16T22:33:02.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673812+0000", 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2136999999999999e-05 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673824+0000", 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673829+0000", 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018687 2020-11-16T22:33:02.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675698+0000", 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9197999999999997e-05 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675767+0000", 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114926 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675882+0000", 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058666569999999999 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.681749+0000", 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032181653999999997 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:02.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713930+0000", 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7389000000000003e-05 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713978+0000", 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.236e-05 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714030+0000", 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62893 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2777088~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.732284748000001, 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001, 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62893 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7876999999999997e-05 2020-11-16T22:33:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674445+0000", 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674452+0000", 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4151999999999999e-05 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674467+0000", 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0669999999999997e-06 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674473+0000", 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066222999999999998 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675135+0000", 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100586 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675236+0000", 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038787833000000001 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714023+0000", 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1931e-05 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714085+0000", 2020-11-16T22:33:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013896399999999999 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714224+0000", 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62894 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2854912~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.73217202, 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997, 2020-11-16T22:33:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62894 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5401000000000002e-05 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674555+0000", 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6089999999999997e-06 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674562+0000", 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0671e-05 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674573+0000", 2020-11-16T22:33:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2869999999999998e-06 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674577+0000", 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068503599999999998 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675262+0000", 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0427000000000003e-05 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675323+0000", 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038926022999999997 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714249+0000", 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3566999999999997e-05 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714302+0000", 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134485 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714437+0000", 2020-11-16T22:33:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.442613704999999, 2020-11-16T22:33:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:03.006 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:03.020+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:03.007 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:33:03.107 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:03.121+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:03.156 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:48779 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 931@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e178)", 2020-11-16T22:33:03.157 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.165763+0000", 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0050516579999999997, 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050761970000000002, 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165763+0000", 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165763+0000", 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3788999999999995e-05 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165827+0000", 2020-11-16T22:33:03.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0797e-05 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165838+0000", 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5654e-05 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165854+0000", 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.366e-06 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165859+0000", 2020-11-16T22:33:03.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4554999999999997e-05 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165893+0000", 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6999000000000001e-05 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165950+0000", 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5605000000000004e-05 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166046+0000", 2020-11-16T22:33:03.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00028276499999999998 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:48787 4.6 4:673a903b:::1000000188d.0000000d:head [trimtrunc 931@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e178)", 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.165915+0000", 2020-11-16T22:33:03.165 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0049001979999999997, 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.005006997, 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165915+0000", 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165915+0000", 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0523999999999999e-05 2020-11-16T22:33:03.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165965+0000", 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6139999999999992e-06 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.168 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165974+0000", 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4440999999999999e-05 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165988+0000", 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5180000000000001e-06 2020-11-16T22:33:03.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.165995+0000", 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010803379999999999 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167075+0000", 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000106681 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167182+0000", 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000170593 2020-11-16T22:33:03.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167353+0000", 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0014377089999999999 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.172 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48778 4.4 e178/170 4:37415b28:::1000000188d.00000004:head v 178'3710, mlcod=178'3708)", 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.166184+0000", 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0046309869999999996, 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0048127650000000001, 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.173 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166184+0000", 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166184+0000", 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4641999999999994e-05 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166269+0000", 2020-11-16T22:33:03.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0078000000000001e-05 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166279+0000", 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5761999999999999e-05 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166294+0000", 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0350000000000004e-06 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166302+0000", 2020-11-16T22:33:03.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1431999999999999e-05 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166343+0000", 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8649999999999998e-05 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.166372+0000", 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018759899999999999 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48780 4.0 e178/161 4:1125f8c3:::1000000188d.00000006:head v 178'9155, mlcod=178'9153)", 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.167361+0000", 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0034541260000000001, 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037052869999999998, 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:03.180 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167361+0000", 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167361+0000", 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5869000000000002e-05 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167437+0000", 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8850000000000005e-06 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167446+0000", 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6920999999999999e-05 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167463+0000", 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8080000000000001e-06 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167468+0000", 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9473e-05 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167508+0000", 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5797999999999999e-05 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167534+0000", 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00017275400000000001 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48783 4.5 e178/117 4:ab64ebfc:::1000000188d.00000009:head v 178'8591, mlcod=178'8589)", 2020-11-16T22:33:03.186 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.167667+0000", 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0031478529999999999, 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0034844580000000002, 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.187 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167667+0000", 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167667+0000", 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5646e-05 2020-11-16T22:33:03.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167743+0000", 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8479999999999996e-06 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167753+0000", 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.0941000000000002e-05 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167774+0000", 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1740000000000003e-06 2020-11-16T22:33:03.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167781+0000", 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00030795300000000002 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168089+0000", 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4428999999999999e-05 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168113+0000", 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00044599100000000002 2020-11-16T22:33:03.192 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48784 4.1 e178/171 4:98a9bd97:::1000000188d.0000000a:head v 178'8607, mlcod=178'8605)", 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.167833+0000", 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0029816759999999999, 2020-11-16T22:33:03.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033847679999999998, 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167833+0000", 2020-11-16T22:33:03.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167833+0000", 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5449000000000002e-05 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167909+0000", 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5740000000000002e-06 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167918+0000", 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5534000000000001e-05 2020-11-16T22:33:03.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167934+0000", 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0559999999999999e-06 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.167941+0000", 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018156199999999999 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168122+0000", 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4964999999999999e-05 2020-11-16T22:33:03.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168147+0000", 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00031414000000000002 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48781 4.4 e178/170 4:3be3f0ca:::1000000188d.00000007:head v 178'3711, mlcod=178'3708)", 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.168485+0000", 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0023299409999999999, 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027978859999999999, 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.200 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168485+0000", 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168485+0000", 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5165000000000003e-05 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168560+0000", 2020-11-16T22:33:03.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.02e-06 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168569+0000", 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3196e-05 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168582+0000", 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7030000000000003e-06 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168588+0000", 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.964e-05 2020-11-16T22:33:03.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168628+0000", 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.0818999999999998e-05 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.168649+0000", 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00016354299999999999 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.206 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48782 4.0 e178/161)", 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.170657+0000", 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00015827199999999999, 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00068923499999999996, 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170657+0000", 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170657+0000", 2020-11-16T22:33:03.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7203e-05 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170724+0000", 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2670000000000006e-06 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170732+0000", 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3047000000000001e-05 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170745+0000", 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4269999999999999e-06 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.170751+0000", 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3943999999999994e-05 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.211 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48786 4.5 e178/117)", 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.171004+0000", 2020-11-16T22:33:03.212 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4294967295.9998112, 2020-11-16T22:33:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039748499999999999, 2020-11-16T22:33:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:03.213 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171004+0000", 2020-11-16T22:33:03.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171004+0000", 2020-11-16T22:33:03.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8959999999999998e-05 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171063+0000", 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1419999999999993e-06 2020-11-16T22:33:03.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171071+0000", 2020-11-16T22:33:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1788e-05 2020-11-16T22:33:03.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171083+0000", 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9189999999999998e-06 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171088+0000", 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3808999999999997e-05 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.220 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:48788 4.4 e178/170)", 2020-11-16T22:33:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:03.171351+0000", 2020-11-16T22:33:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 4294967295.9994645, 2020-11-16T22:33:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010288100000000001, 2020-11-16T22:33:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.422 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171351+0000", 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171351+0000", 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9874999999999998e-05 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171411+0000", 2020-11-16T22:33:03.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7170000000000008e-06 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171418+0000", 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1851000000000001e-05 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171430+0000", 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5560000000000001e-06 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:03.171435+0000", 2020-11-16T22:33:03.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3999000000000007e-05 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 10 2020-11-16T22:33:03.427 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:03.428 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:03.429 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:03.443+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:03.529 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:03.543+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:03.613 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:03.613 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:03.613 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:03.614 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:03.614 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:03.623 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:03.629 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:03.644+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:03.730 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:03.744+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:03.763 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 298.45611366200001, 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 293.45561077399998, 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:33:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:33:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:33:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 286.80965307100001, 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:33:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:33:03.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:33:03.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:33:03.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.792 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 284.56410599100002, 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:33:03.793 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.796 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:33:03.797 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.798 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:33:03.799 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.800 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:33:03.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.802 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 284.56401260500002, 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:33:03.803 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:33:03.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:33:03.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 284.56389220900002, 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:33:03.813 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:33:03.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:33:03.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.822 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 284.56379214200001, 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:33:03.823 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:33:03.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:33:03.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:33:03.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:33:03.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 243.16351915800001, 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.833 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:03.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:03.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:03.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:03.841 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.866744084, 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.843 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:03.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:03.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:03.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:03.851 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 184.56836702499999, 2020-11-16T22:33:03.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:03.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:03.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:03.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:03.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:03.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:03.863 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 123.69141901099999, 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:03.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:03.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:03.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:03.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:03.871 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 118.638821025, 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:03.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:03.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:03.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:03.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:03.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:03.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:03.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:04.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:04.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.083 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 118.637777699, 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:04.084 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:04.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:04.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:04.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.091 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40505 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.104256797000001, 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001, 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:04.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3241e-05 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672457+0000", 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6820000000000002e-06 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672461+0000", 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9270000000000005e-06 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672469+0000", 2020-11-16T22:33:04.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9280000000000003e-06 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672473+0000", 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7399e-05 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672500+0000", 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1418000000000001e-05 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672552+0000", 2020-11-16T22:33:04.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1761000000000006e-05 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672624+0000", 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038091973000000001 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710716+0000", 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00027118299999999998 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710987+0000", 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5528999999999997e-05 2020-11-16T22:33:04.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711032+0000", 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1307999999999999e-05 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711064+0000", 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:04.102 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:04.102 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.104181398000001, 2020-11-16T22:33:04.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:33:04.306 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.306 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.306 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:33:04.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:33:04.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:33:04.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:04.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:33:04.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.103614369999999, 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.316 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:33:04.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:33:04.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:33:04.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62890 4.6 4:6090c0e3:::1000000188d.00000003:head [write 1632508~30811 [387@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.103080151, 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997, 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.324 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62890 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:04.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5053000000000003e-05 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673665+0000", 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9869999999999999e-06 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673670+0000", 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0429999999999996e-06 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673679+0000", 2020-11-16T22:33:04.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2259999999999999e-06 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673683+0000", 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6919999999999998e-05 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673730+0000", 2020-11-16T22:33:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000105 2020-11-16T22:33:04.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673835+0000", 2020-11-16T22:33:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119379 2020-11-16T22:33:04.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673955+0000", 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033775507000000003 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:04.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707730+0000", 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055870140000000004 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713317+0000", 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2094999999999993e-05 2020-11-16T22:33:04.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713389+0000", 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4565000000000001e-05 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713474+0000", 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.538 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62892 4.3 4:db4b432d:::1000000188d.00000005:head [write 2785386~48952 [387@2688279]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:04.539 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.102963752000001, 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001, 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62892 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4876000000000006e-05 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673801+0000", 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2430000000000002e-06 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673807+0000", 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.226e-05 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673820+0000", 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1309999999999997e-06 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673825+0000", 2020-11-16T22:33:04.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3609000000000003e-05 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673858+0000", 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0615999999999997e-05 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673919+0000", 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010215100000000001 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674021+0000", 2020-11-16T22:33:04.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033782754999999998 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707804+0000", 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055369219999999997 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713341+0000", 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3466000000000005e-05 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713404+0000", 2020-11-16T22:33:04.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8748999999999998e-05 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713443+0000", 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.955 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:62889 4.2 e168/163 4:5e663dd9:::1000000188d.00000002:head v 168'1276, mlcod=168'1274)", 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 30.102548932000001, 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998, 2020-11-16T22:33:04.956 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:04.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4456999999999995e-05 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:04.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674216+0000", 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7149999999999998e-06 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674223+0000", 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0988999999999999e-05 2020-11-16T22:33:04.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:04.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:04.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674234+0000", 2020-11-16T22:33:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9389999999999996e-06 2020-11-16T22:33:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674239+0000", 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0126999999999997e-05 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:05.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674279+0000", 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6388999999999998e-05 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674296+0000", 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038984000999999997 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713280+0000", 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1035000000000002e-05 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713331+0000", 2020-11-16T22:33:05.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:05.166 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:05.167 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.979584032000002, 2020-11-16T22:33:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:33:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:05.370 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:05.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:33:05.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:05.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:33:05.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:33:05.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:05.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:05.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:05.378 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:05.378 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:05.379 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:33:05.379 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:05.380 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:33:05.380 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:33:05.381 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:05.382 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:05.393+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:05.480 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:05.495+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:05.534 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:70347 4.0 4:1125f8c3:::1000000188d.00000006:head [write 2996273~19013 [986@0] in=19013b] snapc 1=[] ondisk+write+known_if_redirected e178)", 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:05.544138+0000", 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0055523969999999997, 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055756859999999998, 2020-11-16T22:33:05.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 70347 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544138+0000", 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:05.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544138+0000", 2020-11-16T22:33:05.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6488e-05 2020-11-16T22:33:05.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:05.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544175+0000", 2020-11-16T22:33:05.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2869999999999998e-06 2020-11-16T22:33:05.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544179+0000", 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2439999999999996e-06 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:05.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544186+0000", 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544190+0000", 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1427999999999997e-05 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544231+0000", 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120223 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:05.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544351+0000", 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020638199999999999 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544558+0000", 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000419527 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.547 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:70348 4.1 4:98a9bd97:::1000000188d.0000000a:head [write 864189~59783 [986@0] in=59783b] snapc 1=[] ondisk+write+known_if_redirected e178)", 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:05.544215+0000", 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0054755439999999997, 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055549780000000003, 2020-11-16T22:33:05.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 70348 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544215+0000", 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:05.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544215+0000", 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1275e-05 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544266+0000", 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.76e-06 2020-11-16T22:33:05.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544270+0000", 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0456e-05 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:05.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544281+0000", 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8109999999999999e-06 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544283+0000", 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6241000000000001e-05 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544320+0000", 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9975999999999995e-05 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:05.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544390+0000", 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130455 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544520+0000", 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049635649999999996 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.549484+0000", 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005268539 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:70349 4.0 4:00fd5ff3:::1000000188d.0000000f:head [write 2215936~4096 [986@0] in=4096b] snapc 1=[] ondisk+write+known_if_redirected e178)", 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:05.544321+0000", 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.005370021, 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055035290000000001, 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:05.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 70349 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:05.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544321+0000", 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544321+0000", 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1720000000000001e-05 2020-11-16T22:33:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544352+0000", 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6289999999999999e-06 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544356+0000", 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1350000000000003e-06 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544363+0000", 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.242e-06 2020-11-16T22:33:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.544366+0000", 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005149336 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:05.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.549516+0000", 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9181000000000001e-05 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.549585+0000", 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107885 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:05.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:05.549693+0000", 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053721280000000003 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:05.565 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:05.565 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 3 2020-11-16T22:33:05.565 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:05.566 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:05.566 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:05.570 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:05.570 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:05.571 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:05.571 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:05.571 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:05.573 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:05.574 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:05.581 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:05.596+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:05.682 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:05.696+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:05.709 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:33:05.709 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:33:05.709 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 178, 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:33:03.098969+0000", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:31:03.095061+0000", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:33:05.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5799936, 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:05.711 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:33:05.712 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "jewel", 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:05.714 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:33:05.715 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:05.716 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:05.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:05.719 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:05.720 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:05.721 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:33:05.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:05.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:05.724 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:05.725 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.726 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:05.727 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:05.728 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:05.729 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:05.731 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:33:05.732 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:05.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:05.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:05.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:05.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:05.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:05.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:05.740 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:33:05.741 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:05.743 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:05.744 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:05.745 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:33:05.746 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:05.747 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:05.748 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:05.749 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:05.751 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:05.752 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 0, 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0, 2020-11-16T22:33:05.753 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 119, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 141, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:05.754 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.755 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:33:05.757 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.758 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.759 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:33:05.760 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: "exists" 2020-11-16T22:33:05.762 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 0.463592529296875, 2020-11-16T22:33:05.763 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0.7154998779296875, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 170, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:05.764 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.767 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.768 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.769 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.770 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:33:05.772 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:33:05.773 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 63, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 69, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 108, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 171, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 70, 2020-11-16T22:33:05.774 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:05.776 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:33:05.779 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.780 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:05.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/2959", 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/2959", 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/2959", 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/2959", 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:05.782 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.783 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 141 2020-11-16T22:33:05.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:05.785 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 70 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [], 2020-11-16T22:33:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [ 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "1.0", 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.0", 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.3", 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.5", 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.6", 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.7", 2020-11-16T22:33:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.2", 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.4", 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.6", 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.0", 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.1", 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.2", 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.4", 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.6", 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.800 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:33:05.801 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:33:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:33:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:05.804 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:33:05.805 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:05.805 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:05.805 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:05.806 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:05.819+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:05.807 INFO:tasks.thrashosds.thrasher:j is {'epoch': 178, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:33:03.098969+0000', 'last_up_change': '2020-11-16T22:31:03.095061+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 119, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 0.463592529296875, 'primary_affinity': 0.7154998779296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 170, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 63, 'last_clean_end': 69, 'up_from': 108, 'up_thru': 171, 'down_at': 70, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 2959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 2959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 2959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 2959}]}, 'public_addr': '172.21.15.122:6801/2959', 'cluster_addr': '172.21.15.122:6803/2959', 'heartbeat_back_addr': '172.21.15.122:6807/2959', 'heartbeat_front_addr': '172.21.15.122:6805/2959', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 70}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '1.0', 'osds': [2, 1]}, {'pgid': '2.0', 'osds': [2, 1]}, {'pgid': '2.3', 'osds': [2, 1]}, {'pgid': '2.5', 'osds': [2, 1]}, {'pgid': '2.6', 'osds': [2, 1]}, {'pgid': '2.7', 'osds': [2, 1]}, {'pgid': '3.2', 'osds': [1, 2]}, {'pgid': '3.4', 'osds': [2, 1]}, {'pgid': '3.6', 'osds': [2, 1]}, {'pgid': '4.0', 'osds': [2, 1]}, {'pgid': '4.1', 'osds': [2, 1]}, {'pgid': '4.2', 'osds': [2, 1]}, {'pgid': '4.4', 'osds': [2, 1]}, {'pgid': '4.6', 'osds': [1, 2]}], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:33:05.807 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2020-11-16T22:33:05.905 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:05.920+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:05.995 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:05.995 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.005 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.020+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.006 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:06.099 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:33:06.106 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:06.121+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:06.145 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:06.146 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.155 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:06.238 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:06.253+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:06.308 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.322+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.338 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:06.338 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.349 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:06.408 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.423+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.509 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.523+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~108s 2020-11-16T22:33:06.530 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:06.530 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.540 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:06.610 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:06.625+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:06.701 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:06.701 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.711 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:06.712 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.725+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.812 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:06.827+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:06.906 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:06.907 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:06.913 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:06.927+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:06.916 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:07.014 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:07.028+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:07.109 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:07.109 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:07.115 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:07.129+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:07.119 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:07.215 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:07.230+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:07.262 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:07.262 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:07.271 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:07.316 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:07.331+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:07.417 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:07.431+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:07.418 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:07.418 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:07.428 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:07.517 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:07.532+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:07.575 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:07.615 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:07.615 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:07.618 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:07.632+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:07.626 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:07.718 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:07.733+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:07.725 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:07.746 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:07.746 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:07.746 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:07.746 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:07.748 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:07.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:07.774 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50315 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 1031@2965523] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.781745+0000", 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0060715090000000001, 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060980590000000003, 2020-11-16T22:33:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781745+0000", 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781745+0000", 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6611999999999999e-05 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781772+0000", 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0719999999999999e-06 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781777+0000", 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4600000000000003e-06 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781785+0000", 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781790+0000", 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3485000000000005e-05 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781863+0000", 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2223000000000001e-05 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781955+0000", 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015254999999999999 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782108+0000", 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036249699999999998 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50316 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.782 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.781819+0000", 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0059974370000000004, 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006100584, 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781819+0000", 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781819+0000", 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3277000000000001e-05 2020-11-16T22:33:07.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781843+0000", 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7699999999999999e-06 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781846+0000", 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7190000000000004e-06 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781853+0000", 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2020-11-16T22:33:07.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781856+0000", 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9041000000000001e-05 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781895+0000", 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8298000000000002e-05 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781963+0000", 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116538 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782079+0000", 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026004400000000002 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50317 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.781884+0000", 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.005932377, 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061087290000000002, 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781884+0000", 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781884+0000", 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.527e-05 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781910+0000", 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6610000000000002e-06 2020-11-16T22:33:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781913+0000", 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1160000000000001e-06 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781920+0000", 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.554e-06 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781923+0000", 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046920599999999996 2020-11-16T22:33:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786615+0000", 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100046 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786715+0000", 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000154969 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786870+0000", 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049856759999999996 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.796 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50318 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.781950+0000", 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0058673120000000004, 2020-11-16T22:33:07.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061189749999999996, 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781950+0000", 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781950+0000", 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5609000000000002e-05 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781975+0000", 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4050000000000002e-06 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781980+0000", 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1940000000000001e-06 2020-11-16T22:33:07.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781987+0000", 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3760000000000002e-06 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.781989+0000", 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3203e-05 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782022+0000", 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3357999999999996e-05 2020-11-16T22:33:07.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782116+0000", 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015563299999999999 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782271+0000", 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032177799999999999 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50319 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782027+0000", 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0057894000000000001, 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061130610000000004, 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782027+0000", 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782027+0000", 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9218999999999999e-05 2020-11-16T22:33:07.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782057+0000", 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782061+0000", 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5009999999999998e-06 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782068+0000", 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8100000000000002e-06 2020-11-16T22:33:07.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782071+0000", 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9901999999999998e-05 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782111+0000", 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103559 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782215+0000", 2020-11-16T22:33:07.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183776 2020-11-16T22:33:07.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782398+0000", 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037086200000000001 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:70889 4.0 4.c31fa488 (undecoded) ondisk+write+known_if_redirected e179)", 2020-11-16T22:33:07.812 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782114+0000", 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0057031699999999996, 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060917890000000002, 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:07.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 70889 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782114+0000", 2020-11-16T22:33:07.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782114+0000", 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782153+0000", 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0439999999999998e-06 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782160+0000", 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2553e-05 2020-11-16T22:33:07.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782173+0000", 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9819999999999996e-06 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782178+0000", 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4832999999999994e-05 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50320 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782124+0000", 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0056933280000000001, 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061406289999999999, 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782124+0000", 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782124+0000", 2020-11-16T22:33:07.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.85e-05 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782172+0000", 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9149999999999996e-06 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782180+0000", 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1965e-05 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782192+0000", 2020-11-16T22:33:07.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9809999999999997e-06 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782196+0000", 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040833639999999999 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786279+0000", 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011064400000000001 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786390+0000", 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000182368 2020-11-16T22:33:07.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786572+0000", 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044487370000000004 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50321 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782232+0000", 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.005585085, 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061041799999999998, 2020-11-16T22:33:07.826 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782232+0000", 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782232+0000", 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7026e-05 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782279+0000", 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0000000000000002e-06 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782288+0000", 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3067999999999999e-05 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782301+0000", 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8729999999999998e-06 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782306+0000", 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043658600000000001 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786672+0000", 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4580999999999995e-05 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786756+0000", 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000125327 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786881+0000", 2020-11-16T22:33:07.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046497350000000003 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50323 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782357+0000", 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0054600079999999997, 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060496009999999999, 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782357+0000", 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782357+0000", 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4660999999999998e-05 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782401+0000", 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9360000000000002e-06 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782408+0000", 2020-11-16T22:33:07.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2828e-05 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782421+0000", 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8629999999999999e-06 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782426+0000", 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001694559 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.784121+0000", 2020-11-16T22:33:07.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2349999999999995e-05 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.784213+0000", 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015110500000000001 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.784364+0000", 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020073019999999999 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50324 4.0 4.cffabf00 (undecoded) ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782475+0000", 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0053414439999999999, 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059976279999999996, 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782475+0000", 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782475+0000", 2020-11-16T22:33:07.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6703000000000003e-05 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782522+0000", 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0210000000000002e-06 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782529+0000", 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2173e-05 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782541+0000", 2020-11-16T22:33:07.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5000000000000001e-06 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782546+0000", 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0396999999999997e-05 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:50330 4.3 e179/137 4:cc94663d:::1000000188d.00000015:head v 179'13245, mlcod=179'13243)", 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782530+0000", 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0052864909999999999, 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059946519999999996, 2020-11-16T22:33:07.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782530+0000", 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782530+0000", 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782588+0000", 2020-11-16T22:33:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6819999999999997e-06 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782594+0000", 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.135e-05 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782606+0000", 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.279e-06 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782610+0000", 2020-11-16T22:33:07.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015369489999999999 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.784147+0000", 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1175e-05 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.784168+0000", 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016375980000000001 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50325 4.5 4.ce122cd5 (undecoded) ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782591+0000", 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0052255280000000001, 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059997569999999997, 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782591+0000", 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782591+0000", 2020-11-16T22:33:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3689000000000001e-05 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782635+0000", 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5529999999999998e-06 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782642+0000", 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1404e-05 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782653+0000", 2020-11-16T22:33:07.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0060000000000003e-06 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782657+0000", 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5652000000000001e-05 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50326 4.7 4:fa61569d:::1000000188d.00000011:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782698+0000", 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0051185529999999996, 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059447709999999997, 2020-11-16T22:33:07.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782698+0000", 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782698+0000", 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9909e-05 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782738+0000", 2020-11-16T22:33:07.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1330000000000004e-06 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782744+0000", 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1352000000000001e-05 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782756+0000", 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9029999999999997e-06 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782760+0000", 2020-11-16T22:33:07.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9325e-05 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782799+0000", 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4679999999999996e-05 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782884+0000", 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005582137 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788466+0000", 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6445000000000004e-05 2020-11-16T22:33:07.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788542+0000", 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058438839999999997 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.865 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50327 4.5 4.6a3275 (undecoded) ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782808+0000", 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0050089970000000003, 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059128180000000002, 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.866 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782808+0000", 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782808+0000", 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2227000000000001e-05 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782850+0000", 2020-11-16T22:33:07.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4999999999999996e-06 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782857+0000", 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1770000000000001e-05 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782868+0000", 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1010000000000002e-06 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782872+0000", 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4597999999999994e-05 2020-11-16T22:33:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50328 4.5 4.e44f3815 (undecoded) ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.782915+0000", 2020-11-16T22:33:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0049015589999999998, 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058556989999999998, 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782915+0000", 2020-11-16T22:33:07.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782915+0000", 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2447000000000002e-05 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782958+0000", 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2009999999999996e-06 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782964+0000", 2020-11-16T22:33:07.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1425e-05 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782975+0000", 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7120000000000002e-06 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.782979+0000", 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3785e-05 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.876 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50329 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 1031@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.783020+0000", 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.004796331, 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058065840000000001, 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783020+0000", 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783020+0000", 2020-11-16T22:33:07.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5957000000000001e-05 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783066+0000", 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.998e-06 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783072+0000", 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0927000000000001e-05 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783083+0000", 2020-11-16T22:33:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.703e-06 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783087+0000", 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003181896 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786269+0000", 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8534000000000002e-05 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786368+0000", 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020179099999999999 2020-11-16T22:33:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.786569+0000", 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035488059999999998 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50331 4.5 4.5577df0d (undecoded) ondisk+write+known_if_redirected+full_force e179)", 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.783126+0000", 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0046907069999999997, 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057700989999999999, 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783126+0000", 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783126+0000", 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9480000000000001e-05 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783166+0000", 2020-11-16T22:33:07.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8710000000000002e-06 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783171+0000", 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0757999999999999e-05 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783182+0000", 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5769999999999998e-06 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.783186+0000", 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9685999999999997e-05 2020-11-16T22:33:07.888 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(mds.0.3:50315 4.0 e179/161)", 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:07.788782+0000", 2020-11-16T22:33:07.889 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9990354, 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016422100000000001, 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788782+0000", 2020-11-16T22:33:07.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788782+0000", 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6259e-05 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788808+0000", 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5709999999999997e-06 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788813+0000", 2020-11-16T22:33:07.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1540000000000002e-06 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788822+0000", 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.476e-06 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:07.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:07.788826+0000", 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4459999999999998e-05 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:07.894 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 18 2020-11-16T22:33:07.895 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:07.896 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:07.897 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:07.910+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:07.996 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:08.011+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:08.037 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:08.038 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:08.047 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:08.097 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:08.112+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:08.111 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:33:08.196 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 190.20992598399999, 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:08.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:08.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:08.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:08.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 189.00177581700001, 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.207 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:08.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:08.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:08.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 157.92105996199999, 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:08.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:08.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:08.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.231 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 157.92096452199999, 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:08.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:08.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:08.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:08.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:08.242 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 157.91843019000001, 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:08.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:33:08.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:33:08.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:33:08.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:33:08.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:08.252 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 157.56801039699999, 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:33:08.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:33:08.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:33:08.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:33:08.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:08.262 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 154.86561556000001, 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:33:08.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:33:08.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:33:08.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:33:08.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:08.272 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 128.12510467999999, 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:33:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:33:08.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:33:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:33:08.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:08.282 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 125.66615822200001, 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:08.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:08.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:08.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:08.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:08.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:08.297 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 123.984087241, 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:08.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:33:08.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:33:08.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:33:08.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:33:08.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:08.307 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 123.98364136799999, 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:33:08.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:33:08.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:33:08.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:33:08.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:08.315 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 123.07244430900001, 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:08.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:33:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:33:08.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:33:08.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:33:08.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:08.325 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.291986489, 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:33:08.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:33:08.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:33:08.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:33:08.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 97.185347569000001, 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:08.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:08.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:08.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:08.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:08.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.346 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:40504 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 389@2510211] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.537447020999998, 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998, 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:08.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3415000000000003e-05 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672600+0000", 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8559999999999996e-06 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672609+0000", 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5803000000000001e-05 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672625+0000", 2020-11-16T22:33:08.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0939999999999996e-06 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672631+0000", 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4663999999999998e-05 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672676+0000", 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111723 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672787+0000", 2020-11-16T22:33:08.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015558 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672943+0000", 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063655059999999999 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.679308+0000", 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.034442592000000001 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713751+0000", 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7538000000000002e-05 2020-11-16T22:33:08.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713829+0000", 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2500999999999998e-05 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713881+0000", 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.356 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62889 4.2 4:5e663dd9:::1000000188d.00000002:head [write 2580480~40960 [389@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.536392917999997, 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477, 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62889 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:08.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7012000000000001e-05 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673678+0000", 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.464e-06 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673685+0000", 2020-11-16T22:33:08.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2585999999999999e-05 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673698+0000", 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1329999999999998e-06 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673703+0000", 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180645 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673884+0000", 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1970999999999993e-05 2020-11-16T22:33:08.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673946+0000", 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103269 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674049+0000", 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035807249999999999 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.677630+0000", 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036176901999999997 2020-11-16T22:33:08.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713807+0000", 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1492e-05 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713868+0000", 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129278 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713997+0000", 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477 2020-11-16T22:33:08.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62891 4.5 4:ab64ebfc:::1000000188d.00000009:head [write 132310~6636 [388@0]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.536246231, 2020-11-16T22:33:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002, 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62891 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.368 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7964999999999998e-05 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673806+0000", 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5039999999999999e-06 2020-11-16T22:33:08.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673812+0000", 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2136999999999999e-05 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673824+0000", 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673829+0000", 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018687 2020-11-16T22:33:08.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675698+0000", 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9197999999999997e-05 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675767+0000", 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114926 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675882+0000", 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058666569999999999 2020-11-16T22:33:08.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.681749+0000", 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032181653999999997 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713930+0000", 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7389000000000003e-05 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713978+0000", 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.236e-05 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714030+0000", 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62893 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2777088~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.535586510000002, 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001, 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62893 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7876999999999997e-05 2020-11-16T22:33:08.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674445+0000", 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674452+0000", 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4151999999999999e-05 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674467+0000", 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0669999999999997e-06 2020-11-16T22:33:08.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674473+0000", 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066222999999999998 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675135+0000", 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100586 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675236+0000", 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038787833000000001 2020-11-16T22:33:08.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714023+0000", 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1931e-05 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714085+0000", 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013896399999999999 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714224+0000", 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62894 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2854912~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.535473781999997, 2020-11-16T22:33:08.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997, 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62894 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.388 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5401000000000002e-05 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674555+0000", 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6089999999999997e-06 2020-11-16T22:33:08.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674562+0000", 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0671e-05 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674573+0000", 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2869999999999998e-06 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674577+0000", 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068503599999999998 2020-11-16T22:33:08.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675262+0000", 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0427000000000003e-05 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675323+0000", 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038926022999999997 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714249+0000", 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3566999999999997e-05 2020-11-16T22:33:08.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714302+0000", 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134485 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714437+0000", 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:08.396 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 29.245915467, 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:08.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:08.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:08.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:08.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:08.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:08.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:08.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:08.408 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:08.408 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:08.409 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:08.423+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:08.409 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:08.509 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:08.524+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:08.561 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:50548 4.5 e179/117 4:ab344873:::1000000188d.00000010:head v 179'9086, mlcod=179'9083)", 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:08.572177+0000", 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0046003019999999997, 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00462335, 2020-11-16T22:33:08.562 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572177+0000", 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:08.563 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572177+0000", 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7299000000000006e-05 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:08.564 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572254+0000", 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2939999999999994e-06 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572263+0000", 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.366e-05 2020-11-16T22:33:08.565 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572277+0000", 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4589999999999997e-06 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572283+0000", 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00029270899999999999 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572575+0000", 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5651e-05 2020-11-16T22:33:08.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:08.572591+0000", 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00041407199999999998 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.568 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.569 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:33:08.569 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:33:08.569 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:08.571 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:08.610 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:08.624+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:08.710 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:08.725+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:08.761 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:08.761 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:08.761 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:08.761 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:08.762 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:08.774 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:08.815 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:08.830+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:08.912 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:08.927+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:08.961 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:08.962 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 303.65451253999998, 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:08.963 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:08.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:08.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.967 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:08.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:08.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:08.971 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 298.65400965200001, 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:08.972 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:08.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:08.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:33:08.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:33:08.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:33:08.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:143 3.7 3:f83d65d8:::200.00000004:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:08.981 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 292.00805194899999, 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999, 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:08.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.967037+0000", 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00670676 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973744+0000", 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1930000000000001e-06 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973749+0000", 2020-11-16T22:33:08.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8640000000000008e-06 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973759+0000", 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0169999999999998e-06 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973763+0000", 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3525000000000001e-05 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973806+0000", 2020-11-16T22:33:08.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012448900000000001 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.973931+0000", 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109978 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:16.974041+0000", 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031412048999999997 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.005453+0000", 2020-11-16T22:33:08.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0011918199999999999 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006645+0000", 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7979999999999998e-05 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006693+0000", 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000113075 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:17.006806+0000", 2020-11-16T22:33:08.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039768749999999999 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1113 3.6 3:7bf7e7b5:::200.00000006:head [write 1379046~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:08.991 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.762504869, 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997, 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:08.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212584+0000", 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3921000000000002e-05 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212628+0000", 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3529999999999997e-06 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212632+0000", 2020-11-16T22:33:08.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7539999999999995e-06 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212641+0000", 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3579999999999999e-06 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212645+0000", 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.022733632 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235378+0000", 2020-11-16T22:33:08.996 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000112292 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235490+0000", 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000180867 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.997 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.235671+0000", 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002445483 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238117+0000", 2020-11-16T22:33:08.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0065284649999999998 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244645+0000", 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7265000000000001e-05 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:08.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244703+0000", 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7495000000000002e-05 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244790+0000", 2020-11-16T22:33:09.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032205884999999997 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1114 3.6 3:7bf7e7b5:::200.00000006:head [write 1380431~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:09.001 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.76241148299999, 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997, 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:09.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212678+0000", 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3213000000000006e-05 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212741+0000", 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0549999999999999e-06 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212746+0000", 2020-11-16T22:33:09.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0093e-05 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212756+0000", 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.147e-06 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212761+0000", 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023323654999999999 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236085+0000", 2020-11-16T22:33:09.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000102335 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236187+0000", 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000201628 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236389+0000", 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.001733325 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.238122+0000", 2020-11-16T22:33:09.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0067146669999999997 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244837+0000", 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.634e-05 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244883+0000", 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1205999999999996e-05 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244954+0000", 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032276663999999997 2020-11-16T22:33:09.010 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1115 3.6 3:7bf7e7b5:::200.00000006:head [write 1381816~1385 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:09.011 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.76229108699999, 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001, 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:09.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212798+0000", 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8714999999999997e-05 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212847+0000", 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8280000000000002e-06 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212851+0000", 2020-11-16T22:33:09.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.6459999999999993e-06 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212859+0000", 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3109999999999999e-06 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212863+0000", 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.023894413 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236758+0000", 2020-11-16T22:33:09.016 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000118536 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.236876+0000", 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018908600000000001 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.017 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237065+0000", 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027156350000000001 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.239781+0000", 2020-11-16T22:33:09.018 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0052165459999999999 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.244998+0000", 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1981999999999999e-05 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.019 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245040+0000", 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3966000000000003e-05 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245114+0000", 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032315664000000001 2020-11-16T22:33:09.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:1116 3.6 3:7bf7e7b5:::200.00000006:head [write 1383201~1382 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e88)", 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:09.021 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 289.76219101999999, 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997, 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:09.022 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212898+0000", 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0315000000000003e-05 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.023 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212938+0000", 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1509999999999997e-06 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212942+0000", 2020-11-16T22:33:09.024 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.3990000000000004e-06 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212951+0000", 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0089999999999999e-06 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.025 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.212954+0000", 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.024382029999999999 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237336+0000", 2020-11-16T22:33:09.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.1724999999999994e-05 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237418+0000", 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011051400000000001 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.237528+0000", 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0027040200000000001 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.240232+0000", 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.004924688 2020-11-16T22:33:09.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245157+0000", 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5704999999999997e-05 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245203+0000", 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.0681999999999998e-05 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:19.245283+0000", 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.032385237999999997 2020-11-16T22:33:09.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 248.36191803599999, 2020-11-16T22:33:09.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:09.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:09.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:09.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.037 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:09.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:09.040 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 190.06514296200001, 2020-11-16T22:33:09.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:09.042 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.043 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.045 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:09.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:09.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.050 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 189.76676590299999, 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:09.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:09.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.055 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:09.056 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.057 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:09.058 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.059 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:09.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 128.889817889, 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.062 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.063 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:09.064 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.065 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:09.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.067 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:09.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 123.837219903, 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.070 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:09.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:09.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:09.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.077 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 123.836176577, 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.078 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:09.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:09.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:09.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40505 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.302655674999997, 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001, 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.086 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672433+0000", 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3241e-05 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672457+0000", 2020-11-16T22:33:09.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6820000000000002e-06 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672461+0000", 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9270000000000005e-06 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672469+0000", 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9280000000000003e-06 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672473+0000", 2020-11-16T22:33:09.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.7399e-05 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672500+0000", 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1418000000000001e-05 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672552+0000", 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1761000000000006e-05 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672624+0000", 2020-11-16T22:33:09.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038091973000000001 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710716+0000", 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00027118299999999998 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.710987+0000", 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5528999999999997e-05 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711032+0000", 2020-11-16T22:33:09.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1307999999999999e-05 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.711064+0000", 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038630349000000001 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.095 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.302580276, 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.096 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:33:09.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:33:09.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:33:09.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:33:09.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.302013248000002, 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.106 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:33:09.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.109 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:33:09.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:33:09.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62890 4.6 4:6090c0e3:::1000000188d.00000003:head [write 1632508~30811 [387@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.301479028999999, 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997, 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.114 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62890 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673610+0000", 2020-11-16T22:33:09.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5053000000000003e-05 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673665+0000", 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9869999999999999e-06 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.117 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673670+0000", 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0429999999999996e-06 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673679+0000", 2020-11-16T22:33:09.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2259999999999999e-06 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673683+0000", 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6919999999999998e-05 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673730+0000", 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000105 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673835+0000", 2020-11-16T22:33:09.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000119379 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673955+0000", 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033775507000000003 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707730+0000", 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055870140000000004 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713317+0000", 2020-11-16T22:33:09.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2094999999999993e-05 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713389+0000", 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4565000000000001e-05 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713474+0000", 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039863788999999997 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.124 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:62892 4.3 4:db4b432d:::1000000188d.00000005:head [write 2785386~48952 [387@2688279]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.30136263, 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001, 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:09.125 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 62892 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:09.126 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673726+0000", 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4876000000000006e-05 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.127 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673801+0000", 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2430000000000002e-06 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673807+0000", 2020-11-16T22:33:09.128 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.226e-05 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673820+0000", 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1309999999999997e-06 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673825+0000", 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3609000000000003e-05 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673858+0000", 2020-11-16T22:33:09.130 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0615999999999997e-05 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673919+0000", 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010215100000000001 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.131 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674021+0000", 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.033782754999999998 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707804+0000", 2020-11-16T22:33:09.132 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0055369219999999997 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713341+0000", 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3466000000000005e-05 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.133 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713404+0000", 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8748999999999998e-05 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713443+0000", 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039716778000000001 2020-11-16T22:33:09.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:62889 4.2 e168/163 4:5e663dd9:::1000000188d.00000002:head v 168'1276, mlcod=168'1274)", 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:09.135 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 35.300947809999997, 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998, 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:09.136 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674141+0000", 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4456999999999995e-05 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.137 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674216+0000", 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7149999999999998e-06 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674223+0000", 2020-11-16T22:33:09.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0988999999999999e-05 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674234+0000", 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9389999999999996e-06 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674239+0000", 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0126999999999997e-05 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674279+0000", 2020-11-16T22:33:09.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6388999999999998e-05 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.674296+0000", 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038984000999999997 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713280+0000", 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1035000000000002e-05 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713331+0000", 2020-11-16T22:33:09.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.039189651999999998 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:09.143 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.177982909999997, 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:33:09.144 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.145 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:33:09.146 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.147 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:33:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:33:09.150 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.151 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.151 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.151 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:09.151 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:09.151 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:09.152 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:33:09.153 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:09.153 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:09.167+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:09.253 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:09.268+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:09.315 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:09.315 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:09.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:50752 3.5 3:bec9163d:::200.00000015:head [write 1303374~1923 [fadvise_dontneed] in=1923b] snapc 0=[] ondisk+write+known_if_redirected+full_force e180)", 2020-11-16T22:33:09.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:09.327265+0000", 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0030033429999999999, 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030245770000000001, 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327265+0000", 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327265+0000", 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7925e-05 2020-11-16T22:33:09.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327303+0000", 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6080000000000003e-06 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327309+0000", 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2006e-05 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327321+0000", 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:09.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327326+0000", 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3312000000000002e-05 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327360+0000", 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3713000000000005e-05 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.327423+0000", 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026912170000000001 2020-11-16T22:33:09.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.330115+0000", 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4788000000000003e-05 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:09.330189+0000", 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029244520000000001 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:09.323 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:33:09.324 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:09.325 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:09.354 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:09.369+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:09.454 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:09.469+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:09.518 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:09.518 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:09.518 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:09.518 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:09.518 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:09.529 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:09.555 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:09.570+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:09.655 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:09.670+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:09.727 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:09.728 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:09.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:09.728 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:09.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.728 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 191.740805196, 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:09.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:09.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:09.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:09.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:09.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 190.53265502900001, 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:09.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:09.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:09.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:09.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:09.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 159.45193917399999, 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:09.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:09.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:09.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:09.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:09.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:09.967 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 159.45184373399999, 2020-11-16T22:33:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:10.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:10.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:10.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 159.44930940200001, 2020-11-16T22:33:10.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:10.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:33:10.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:33:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:33:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:33:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:33:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:10.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:33:10.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:33:10.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:10.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 159.098889609, 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:33:10.800 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:10.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:33:10.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:33:10.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:10.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:11.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:33:11.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:33:11.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:11.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:33:11.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:33:11.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.012 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 156.39649477200001, 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:33:11.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:11.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:11.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:33:11.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:33:11.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:33:11.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:33:11.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:11.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:33:11.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:33:11.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:11.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.428 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 129.65598389199999, 2020-11-16T22:33:11.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:11.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:11.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:11.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:33:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:33:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:33:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:33:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:33:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:33:11.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 127.19703743399999, 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:11.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:11.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:11.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:11.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:11.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:11.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:12.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:12.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:12.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:12.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.059 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:12.262 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:12.262 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 125.514966453, 2020-11-16T22:33:12.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:33:12.262 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:12.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:33:12.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:12.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:33:12.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:33:12.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:33:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:33:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:33:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:33:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:12.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 125.51452058, 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:12.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:12.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:33:12.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:33:12.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:12.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:12.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 124.60332352099999, 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:12.687 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:12.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:12.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:33:12.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:12.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:33:12.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:12.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:12.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:13.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:33:13.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:33:13.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:13.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:33:13.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:13.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:33:13.103 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:13.103 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.822865701, 2020-11-16T22:33:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:33:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:33:13.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:33:13.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:33:13.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:13.315 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.716226781000003, 2020-11-16T22:33:13.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:13.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:13.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:13.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:13.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:13.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:13.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:13.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:13.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.935 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:40504 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 389@2510211] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.068326233000001, 2020-11-16T22:33:13.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998, 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3415000000000003e-05 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:13.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:13.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672600+0000", 2020-11-16T22:33:13.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8559999999999996e-06 2020-11-16T22:33:13.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672609+0000", 2020-11-16T22:33:14.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5803000000000001e-05 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672625+0000", 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0939999999999996e-06 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672631+0000", 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4663999999999998e-05 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672676+0000", 2020-11-16T22:33:14.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111723 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672787+0000", 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015558 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672943+0000", 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063655059999999999 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.679308+0000", 2020-11-16T22:33:14.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.034442592000000001 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713751+0000", 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7538000000000002e-05 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713829+0000", 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2500999999999998e-05 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713881+0000", 2020-11-16T22:33:14.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62889 4.2 4:5e663dd9:::1000000188d.00000002:head [write 2580480~40960 [389@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:14.149 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.067272129999999, 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477, 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62889 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:14.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:14.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7012000000000001e-05 2020-11-16T22:33:14.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673678+0000", 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.464e-06 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673685+0000", 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2585999999999999e-05 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673698+0000", 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1329999999999998e-06 2020-11-16T22:33:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673703+0000", 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180645 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673884+0000", 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1970999999999993e-05 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673946+0000", 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103269 2020-11-16T22:33:14.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674049+0000", 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035807249999999999 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:14.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.677630+0000", 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036176901999999997 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713807+0000", 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1492e-05 2020-11-16T22:33:14.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713868+0000", 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129278 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713997+0000", 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62891 4.5 4:ab64ebfc:::1000000188d.00000009:head [write 132310~6636 [388@0]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:14.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.067125443000002, 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002, 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:14.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62891 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:14.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673748+0000", 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7964999999999998e-05 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673806+0000", 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5039999999999999e-06 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673812+0000", 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2136999999999999e-05 2020-11-16T22:33:14.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673824+0000", 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673829+0000", 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018687 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675698+0000", 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9197999999999997e-05 2020-11-16T22:33:14.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675767+0000", 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114926 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675882+0000", 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058666569999999999 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.681749+0000", 2020-11-16T22:33:14.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.032181653999999997 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713930+0000", 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7389000000000003e-05 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713978+0000", 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.236e-05 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714030+0000", 2020-11-16T22:33:14.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040282389000000002 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62893 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2777088~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:14.373 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.066465721999997, 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001, 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:14.374 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62893 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674407+0000", 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7876999999999997e-05 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674445+0000", 2020-11-16T22:33:14.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674452+0000", 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4151999999999999e-05 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674467+0000", 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0669999999999997e-06 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674473+0000", 2020-11-16T22:33:14.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066222999999999998 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675135+0000", 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100586 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675236+0000", 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038787833000000001 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714023+0000", 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1931e-05 2020-11-16T22:33:14.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714085+0000", 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013896399999999999 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714224+0000", 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039816902000000001 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.382 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62894 4.2 4:5e663dd9:::1000000188d.00000002:head [read 2854912~53248 [389@4194304]] snapc 0=[] ondisk+read+known_if_redirected e168)", 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.066352993999999, 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997, 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62894 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:14.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674520+0000", 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5401000000000002e-05 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674555+0000", 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6089999999999997e-06 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674562+0000", 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0671e-05 2020-11-16T22:33:14.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674573+0000", 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2869999999999998e-06 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674577+0000", 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068503599999999998 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675262+0000", 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0427000000000003e-05 2020-11-16T22:33:14.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.675323+0000", 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.038926022999999997 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714249+0000", 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3566999999999997e-05 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714302+0000", 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134485 2020-11-16T22:33:14.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.714437+0000", 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.039916505999999997 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.776794679000002, 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:14.392 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:14.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:14.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:14.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:14.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:14.604 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:14.604 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:14.605 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:14.605 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:33:14.606 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:33:14.606 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:33:14.608 INFO:tasks.workunit.client.0.smithi138.stdout:20000 punch from 0x167a970 to 0x1681166, (0x67f6 bytes) 2020-11-16T22:33:14.608 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:14.620+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:14.609 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:14.706 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:14.721+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:14.751 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:14.752 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:14.752 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:33:14.752 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:14.762 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:14.768 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:14.789 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:14.789 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:14.789 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:14.789 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:14.789 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:14.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:14.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:14.844 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:14.859+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:14.908 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:14.922+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:14.933 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:33:14.935 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:33:14.946 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2020-11-16T22:33:13.755327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136345518,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":2256,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":43198,"num_read_kb":1481841,"num_write":85883,"num_write_kb":832836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6804,"num_bytes_recovered":139054382,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75761,"ondisk_log_size":75761,"up":28,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":690228,"kb_used_data":201264,"kb_used_omap":26,"kb_used_meta":488613,"kb_avail":282425292,"statfs":{"total":289910292480,"available":289203499008,"internally_reserved":0,"allocated":206094336,"data_stored":247139471,"data_compressed":19866486,"data_compressed_allocated":68399104,"data_compressed_original":136778808,"omap_allocated":27325,"internal_metadata":500340035},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14612688,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1360,"num_read_kb":32687,"num_write":3572,"num_write_kb":20445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001731"},"pg_stats":[{"pgid":"4.2","version":"180'1790","reported_seq":"2502","reported_epoch":"180","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:12.138674+0000","last_change":"2020-11-16T22:32:04.140287+0000","last_active":"2020-11-16T22:33:12.138674+0000","last_peered":"2020-11-16T22:33:12.138674+0000","last_clean":"2020-11-16T22:33:12.138674+0000","last_became_active":"2020-11-16T22:32:04.136650+0000","last_became_peered":"2020-11-16T22:32:04.136650+0000","last_unstale":"2020-11-16T22:33:12.138674+0000","last_undegraded":"2020-11-16T22:33:12.138674+0000","last_fullsized":"2020-11-16T22:33:12.138674+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1790,"ondisk_log_size":1790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6924207,"num_objects":457,"num_object_clones":0,"num_object_copies":914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":457,"num_objects_unfound":0,"num_objects_dirty":457,"num_whiteouts":0,"num_read":568,"num_read_kb":21730,"num_write":1914,"num_write_kb":17586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"46","reported_epoch":"179","state":"active+undersized","last_fresh":"2020-11-16T22:33:06.114362+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:33:06.114362+0000","last_peered":"2020-11-16T22:33:06.114362+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:33:06.114362+0000","last_undegraded":"2020-11-16T22:33:06.114362+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'4124","reported_seq":"4281","reported_epoch":"180","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:12.156329+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:33:12.156329+0000","last_peered":"2020-11-16T22:33:12.156329+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:33:12.156329+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4124,"ondisk_log_size":4124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14372259,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4121,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"180'4264","reported_seq":"9069","reported_epoch":"180","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:12.167740+0000","last_change":"2020-11-16T22:32:38.960533+0000","last_active":"2020-11-16T22:33:12.167740+0000","last_peered":"2020-11-16T22:33:12.167740+0000","last_clean":"2020-11-16T22:33:12.167740+0000","last_became_active":"2020-11-16T22:32:38.951761+0000","last_became_peered":"2020-11-16T22:32:38.951761+0000","last_unstale":"2020-11-16T22:33:12.167740+0000","last_undegraded":"2020-11-16T22:33:12.167740+0000","last_fullsized":"2020-11-16T22:33:12.167740+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4264,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7838976,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":447,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":1959,"num_read_kb":65233,"num_write":4320,"num_write_kb":27240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"109","reported_epoch":"179","state":"active+undersized","last_fresh":"2020-11-16T22:33:06.114062+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:33:06.114062+0000","last_peered":"2020-11-16T22:33:06.114062+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:33:06.114062+0000","last_undegraded":"2020-11-16T22:33:06.114062+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"168'4360","reported_seq":"4519","reported_epoch":"179","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:06.114260+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:33:06.114260+0000","last_peered":"2020-11-16T22:33:06.114260+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:33:06.114260+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4360,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"180'2249","reported_seq":"3235","reported_epoch":"180","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:12.094708+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:33:12.094708+0000","last_peered":"2020-11-16T22:33:12.094708+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:33:12.094708+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":2249,"ondisk_log_size":2249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6041692,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":791,"num_read_kb":29361,"num_write":2351,"num_write_kb":14145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":451}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"46","reported_epoch":"179","state":"active+undersized","last_fresh":"2020-11-16T22:33:06.114227+0000","last_change":"2020-11-16T22:31:04.115597+0000","last_active":"2020-11-16T22:33:06.114227+0000","last_peered":"2020-11-16T22:33:06.114227+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115597+0000","last_became_peered":"2020-11-16T22:31:04.115597+0000","last_unstale":"2020-11-16T22:33:06.114227+0000","last_undegraded":"2020-11-16T22:33:06.114227+0000","last_fullsized":"2020-11-16T22:31:04.110788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"165'10812","reported_seq":"11006","reported_epoch":"179","state":"active+clean","last_fresh":"2020-11-16T22:33:06.113909+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:33:06.113909+0000","last_peered":"2020-11-16T22:33:06.113909+0000","last_clean":"2020-11-16T22:33:06.113909+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:33:06.113909+0000","last_undegraded":"2020-11-16T22:33:06.113909+0000","last_fullsized":"2020-11-16T22:33:06.113909+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10012,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"180'9386","reported_seq":"13415","reported_epoch":"180","state":"active+clean","last_fresh":"2020-11-16T22:33:12.130719+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:33:12.130719+0000","last_peered":"2020-11-16T22:33:12.130719+0000","last_clean":"2020-11-16T22:33:12.130719+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:33:12.130719+0000","last_undegraded":"2020-11-16T22:33:12.130719+0000","last_fullsized":"2020-11-16T22:33:12.130719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9386,"ondisk_log_size":9386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2924420,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":3865,"num_read_kb":95629,"num_write":9463,"num_write_kb":85948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"131","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.113946+0000","last_change":"2020-11-16T22:32:02.128340+0000","last_active":"2020-11-16T22:33:06.113946+0000","last_peered":"2020-11-16T22:33:06.113946+0000","last_clean":"2020-11-16T22:33:06.113946+0000","last_became_active":"2020-11-16T22:32:02.127967+0000","last_became_peered":"2020-11-16T22:32:02.127967+0000","last_unstale":"2020-11-16T22:33:06.113946+0000","last_undegraded":"2020-11-16T22:33:06.113946+0000","last_fullsized":"2020-11-16T22:33:06.113946+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"117","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.113671+0000","last_change":"2020-11-16T22:32:39.953425+0000","last_active":"2020-11-16T22:33:06.113671+0000","last_peered":"2020-11-16T22:33:06.113671+0000","last_clean":"2020-11-16T22:33:06.113671+0000","last_became_active":"2020-11-16T22:32:39.952851+0000","last_became_peered":"2020-11-16T22:32:39.952851+0000","last_unstale":"2020-11-16T22:33:06.113671+0000","last_undegraded":"2020-11-16T22:33:06.113671+0000","last_fullsized":"2020-11-16T22:33:06.113671+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"165'4341","reported_seq":"4484","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.114227+0000","last_change":"2020-11-16T22:32:02.131160+0000","last_active":"2020-11-16T22:33:06.114227+0000","last_peered":"2020-11-16T22:33:06.114227+0000","last_clean":"2020-11-16T22:33:06.114227+0000","last_became_active":"2020-11-16T22:32:02.129584+0000","last_became_peered":"2020-11-16T22:32:02.129584+0000","last_unstale":"2020-11-16T22:33:06.114227+0000","last_undegraded":"2020-11-16T22:33:06.114227+0000","last_fullsized":"2020-11-16T22:33:06.114227+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4341,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":12835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"180'2269","reported_seq":"4587","reported_epoch":"180","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:09.698283+0000","last_change":"2020-11-16T22:32:02.131605+0000","last_active":"2020-11-16T22:33:09.698283+0000","last_peered":"2020-11-16T22:33:09.698283+0000","last_clean":"2020-11-16T22:33:09.698283+0000","last_became_active":"2020-11-16T22:32:02.127816+0000","last_became_peered":"2020-11-16T22:32:02.127816+0000","last_unstale":"2020-11-16T22:33:09.698283+0000","last_undegraded":"2020-11-16T22:33:09.698283+0000","last_fullsized":"2020-11-16T22:33:09.698283+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6106585,"num_objects":438,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":438,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":820,"num_read_kb":24116,"num_write":2358,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":438}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"113","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.113667+0000","last_change":"2020-11-16T22:32:38.951861+0000","last_active":"2020-11-16T22:33:06.113667+0000","last_peered":"2020-11-16T22:33:06.113667+0000","last_clean":"2020-11-16T22:33:06.113667+0000","last_became_active":"2020-11-16T22:32:38.951148+0000","last_became_peered":"2020-11-16T22:32:38.951148+0000","last_unstale":"2020-11-16T22:33:06.113667+0000","last_undegraded":"2020-11-16T22:33:06.113667+0000","last_fullsized":"2020-11-16T22:33:06.113667+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"220","reported_epoch":"179","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:06.113730+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:33:06.113730+0000","last_peered":"2020-11-16T22:33:06.113730+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:33:06.113730+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"180'13304","reported_seq":"29267","reported_epoch":"180","state":"active+clean","last_fresh":"2020-11-16T22:33:09.700491+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:33:09.700491+0000","last_peered":"2020-11-16T22:33:09.700491+0000","last_clean":"2020-11-16T22:33:09.700491+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:33:09.700491+0000","last_undegraded":"2020-11-16T22:33:09.700491+0000","last_fullsized":"2020-11-16T22:33:09.700491+0000","mapping_epoch":137,"log_start":"94'3300","ondisk_log_start":"94'3300","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10004,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7497620,"num_objects":466,"num_object_clones":0,"num_object_copies":932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":466,"num_whiteouts":0,"num_read":14865,"num_read_kb":369309,"num_write":17190,"num_write_kb":298887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"89","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.113977+0000","last_change":"2020-11-16T22:32:39.955775+0000","last_active":"2020-11-16T22:33:06.113977+0000","last_peered":"2020-11-16T22:33:06.113977+0000","last_clean":"2020-11-16T22:33:06.113977+0000","last_became_active":"2020-11-16T22:32:39.955217+0000","last_became_peered":"2020-11-16T22:32:39.955217+0000","last_unstale":"2020-11-16T22:33:06.113977+0000","last_undegraded":"2020-11-16T22:33:06.113977+0000","last_fullsized":"2020-11-16T22:33:06.113977+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"177'2237","reported_seq":"2232","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.114093+0000","last_change":"2020-11-16T22:32:39.957132+0000","last_active":"2020-11-16T22:33:06.114093+0000","last_peered":"2020-11-16T22:33:06.114093+0000","last_clean":"2020-11-16T22:33:06.114093+0000","last_became_active":"2020-11-16T22:32:39.955410+0000","last_became_peered":"2020-11-16T22:32:39.955410+0000","last_unstale":"2020-11-16T22:33:06.114093+0000","last_undegraded":"2020-11-16T22:33:06.114093+0000","last_fullsized":"2020-11-16T22:33:06.114093+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2237,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389251,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":6}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"180'9542","reported_seq":"21908","reported_epoch":"180","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:12.152926+0000","last_change":"2020-11-16T22:32:02.133514+0000","last_active":"2020-11-16T22:33:12.152926+0000","last_peered":"2020-11-16T22:33:12.152926+0000","last_clean":"2020-11-16T22:33:12.152926+0000","last_became_active":"2020-11-16T22:32:02.127854+0000","last_became_peered":"2020-11-16T22:32:02.127854+0000","last_unstale":"2020-11-16T22:33:12.152926+0000","last_undegraded":"2020-11-16T22:33:12.152926+0000","last_fullsized":"2020-11-16T22:33:12.152926+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9542,"ondisk_log_size":9542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6873901,"num_objects":439,"num_object_clones":0,"num_object_copies":878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":9447,"num_read_kb":386852,"num_write":10100,"num_write_kb":136309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"144'9","reported_seq":"95","reported_epoch":"179","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:06.114774+0000","last_change":"2020-11-16T22:31:04.115488+0000","last_active":"2020-11-16T22:33:06.114774+0000","last_peered":"2020-11-16T22:33:06.114774+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:33:06.114774+0000","last_undegraded":"2020-11-16T22:31:04.111159+0000","last_fullsized":"2020-11-16T22:31:04.111040+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"95","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.113861+0000","last_change":"2020-11-16T22:32:38.951880+0000","last_active":"2020-11-16T22:33:06.113861+0000","last_peered":"2020-11-16T22:33:06.113861+0000","last_clean":"2020-11-16T22:33:06.113861+0000","last_became_active":"2020-11-16T22:32:38.951168+0000","last_became_peered":"2020-11-16T22:32:38.951168+0000","last_unstale":"2020-11-16T22:33:06.113861+0000","last_undegraded":"2020-11-16T22:33:06.113861+0000","last_fullsized":"2020-11-16T22:33:06.113861+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"180'8872","reported_seq":"20106","reported_epoch":"180","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:12.116370+0000","last_change":"2020-11-16T22:32:39.965767+0000","last_active":"2020-11-16T22:33:12.116370+0000","last_peered":"2020-11-16T22:33:12.116370+0000","last_clean":"2020-11-16T22:33:12.116370+0000","last_became_active":"2020-11-16T22:32:39.955272+0000","last_became_peered":"2020-11-16T22:32:39.955272+0000","last_unstale":"2020-11-16T22:33:12.116370+0000","last_undegraded":"2020-11-16T22:33:12.116370+0000","last_fullsized":"2020-11-16T22:33:12.116370+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":8872,"ondisk_log_size":8872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6243682,"num_objects":458,"num_object_clones":0,"num_object_copies":916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":458,"num_objects_unfound":0,"num_objects_dirty":458,"num_whiteouts":0,"num_read":10857,"num_read_kb":489380,"num_write":10009,"num_write_kb":150766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":458}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"113","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.114248+0000","last_change":"2020-11-16T22:32:38.951920+0000","last_active":"2020-11-16T22:33:06.114248+0000","last_peered":"2020-11-16T22:33:06.114248+0000","last_clean":"2020-11-16T22:33:06.114248+0000","last_became_active":"2020-11-16T22:32:38.951358+0000","last_became_peered":"2020-11-16T22:32:38.951358+0000","last_unstale":"2020-11-16T22:33:06.114248+0000","last_undegraded":"2020-11-16T22:33:06.114248+0000","last_fullsized":"2020-11-16T22:33:06.114248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"178'2270","reported_seq":"2316","reported_epoch":"179","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:06.114229+0000","last_change":"2020-11-16T22:32:02.132544+0000","last_active":"2020-11-16T22:33:06.114229+0000","last_peered":"2020-11-16T22:33:06.114229+0000","last_clean":"2020-11-16T22:33:06.114229+0000","last_became_active":"2020-11-16T22:32:02.132057+0000","last_became_peered":"2020-11-16T22:32:02.132057+0000","last_unstale":"2020-11-16T22:33:06.114229+0000","last_undegraded":"2020-11-16T22:33:06.114229+0000","last_fullsized":"2020-11-16T22:33:06.114229+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2270,"ondisk_log_size":2270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":50451083,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":43172,"num_read_kb":1481610,"num_write":57705,"num_write_kb":744929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76509184,"data_stored":62707392,"data_compressed":5616527,"data_compressed_allocated":10899456,"data_compressed_original":21779512,"omap_allocated":0,"internal_metadata":0},"log_size":48376,"ondisk_log_size":48376,"up":10,"acting":15,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":85894435,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":13,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":28174,"num_write_kb":87887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":106481772,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125104128,"data_stored":182583238,"data_compressed":14251271,"data_compressed_allocated":57548800,"data_compressed_original":115097600,"omap_allocated":10534,"internal_metadata":0},"log_size":27381,"ondisk_log_size":27381,"up":9,"acting":12,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":1,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":13,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260528,"kb_used_data":89000,"kb_used_omap":26,"kb_used_meta":171493,"kb_avail":94111312,"statfs":{"total":96636764160,"available":96369983488,"internally_reserved":0,"allocated":91136000,"data_stored":110455286,"data_compressed":8750486,"data_compressed_allocated":30756864,"data_compressed_original":61506922,"omap_allocated":27325,"internal_metadata":175609155},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:33:00 2020","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.57599999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73799999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109904,"kb_used_data":35336,"kb_used_omap":0,"kb_used_meta":74560,"kb_avail":94261936,"statfs":{"total":96636764160,"available":96524222464,"internally_reserved":0,"allocated":36184064,"data_stored":55423993,"data_compressed":6804590,"data_compressed_allocated":25083904,"data_compressed_original":50167808,"omap_allocated":0,"internal_metadata":76349440},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:30:20 2020","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":1.028},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738490,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319796,"kb_used_data":76928,"kb_used_omap":0,"kb_used_meta":242560,"kb_avail":94052044,"statfs":{"total":96636764160,"available":96309293056,"internally_reserved":0,"allocated":78774272,"data_stored":81260192,"data_compressed":4311410,"data_compressed_allocated":12558336,"data_compressed_original":25104078,"omap_allocated":0,"internal_metadata":248381440},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:33:01 2020","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.998,"15min":0.998},"last":0.439},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47199999999999998}]}]}]}} 2020-11-16T22:33:17.404 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:17.405 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:17.406 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:17.406 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:17.407 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:17.419+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:17.407 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:33:17.505 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:17.519+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:17.585 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:17.586 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:17.592 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:17.596 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:17.606 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:17.620+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:17.615 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:17.615 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:17.615 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:17.615 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:17.617 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:17.617 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:17.707 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:17.722+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:17.743 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:17.743 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:17.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53321 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 1230@1069261] snapc 1=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.751933+0000", 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0058885500000000002, 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059078639999999997, 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751933+0000", 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751933+0000", 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6271999999999999e-05 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751960+0000", 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4179999999999996e-06 2020-11-16T22:33:17.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751965+0000", 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2820000000000002e-06 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751974+0000", 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2919999999999997e-06 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.751979+0000", 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2579e-05 2020-11-16T22:33:17.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752021+0000", 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:17.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752128+0000", 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173041 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752301+0000", 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041241890000000003 2020-11-16T22:33:17.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:17.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.756425+0000", 2020-11-16T22:33:17.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044920730000000001 2020-11-16T22:33:17.954 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53322 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1230@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752026+0000", 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.005795967, 2020-11-16T22:33:17.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058659259999999996, 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752026+0000", 2020-11-16T22:33:17.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752026+0000", 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1926999999999999e-05 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752078+0000", 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3100000000000003e-06 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752085+0000", 2020-11-16T22:33:17.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2743e-05 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752098+0000", 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9470000000000002e-06 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752103+0000", 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6893000000000001e-05 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752140+0000", 2020-11-16T22:33:17.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5042999999999994e-05 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752205+0000", 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013950200000000001 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752344+0000", 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031836499999999998 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:17.962 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53323 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 1230@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752158+0000", 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0056644110000000003, 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057767440000000003, 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:17.963 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:18.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:18.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752158+0000", 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752158+0000", 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6867999999999999e-05 2020-11-16T22:33:18.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752204+0000", 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0010000000000004e-06 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:18.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752211+0000", 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2636000000000001e-05 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752224+0000", 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.622e-06 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752229+0000", 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7097e-05 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:18.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752276+0000", 2020-11-16T22:33:18.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000104721 2020-11-16T22:33:18.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:18.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752380+0000", 2020-11-16T22:33:18.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014678 2020-11-16T22:33:18.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752527+0000", 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000369725 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:18.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:73202 4.5 4:a7565345:::1000000188d.0000000c:head [zero 232424~8506] snapc 1=[] ondisk+write+known_if_redirected e181)", 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752245+0000", 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0055766219999999998, 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057350680000000003, 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:18.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 73202 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:18.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752245+0000", 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:18.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752245+0000", 2020-11-16T22:33:18.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8772000000000002e-05 2020-11-16T22:33:18.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752274+0000", 2020-11-16T22:33:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2159999999999997e-06 2020-11-16T22:33:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752279+0000", 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3171e-05 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752292+0000", 2020-11-16T22:33:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3900000000000003e-06 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752297+0000", 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048954899999999997 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757192+0000", 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6116999999999998e-05 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757248+0000", 2020-11-16T22:33:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9061999999999994e-05 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757348+0000", 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051022180000000004 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53325 4.5 4.6a3275 (undecoded) ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752413+0000", 2020-11-16T22:33:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0054090850000000001, 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056504759999999998, 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752413+0000", 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752413+0000", 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5726999999999998e-05 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752459+0000", 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9299999999999997e-06 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752465+0000", 2020-11-16T22:33:18.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2298e-05 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752478+0000", 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.442e-06 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752482+0000", 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9397e-05 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:53328 4.3 e181/137 4:cc94663d:::1000000188d.00000015:head v 181'13558, mlcod=181'13556)", 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752489+0000", 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.005333295, 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056302289999999996, 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:18.796 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752489+0000", 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:18.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752489+0000", 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6297999999999994e-05 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752555+0000", 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8140000000000007e-06 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752564+0000", 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2204999999999999e-05 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:18.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752576+0000", 2020-11-16T22:33:18.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:33:19.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:19.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752581+0000", 2020-11-16T22:33:19.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5621999999999998e-05 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752627+0000", 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5794000000000001e-05 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752653+0000", 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016395400000000001 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53326 4.5 4.e44f3815 (undecoded) ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752528+0000", 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0052943770000000003, 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056580190000000002, 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:19.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752528+0000", 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:19.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752528+0000", 2020-11-16T22:33:19.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3837000000000001e-05 2020-11-16T22:33:19.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:19.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752571+0000", 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9419999999999998e-06 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752578+0000", 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2508e-05 2020-11-16T22:33:19.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752591+0000", 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2270000000000004e-06 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752595+0000", 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7514000000000004e-05 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53327 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 1230@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752639+0000", 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0051824990000000001, 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056034400000000003, 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:19.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:19.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752639+0000", 2020-11-16T22:33:19.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752639+0000", 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1915000000000001e-05 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752681+0000", 2020-11-16T22:33:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3980000000000004e-06 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752688+0000", 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752699+0000", 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1450000000000001e-06 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752703+0000", 2020-11-16T22:33:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015090100000000001 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.754212+0000", 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105728 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.754318+0000", 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016280900000000001 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.754481+0000", 2020-11-16T22:33:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018411689999999999 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53329 4.5 4.5577df0d (undecoded) ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.752757+0000", 2020-11-16T22:33:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0050647180000000002, 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005560929, 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752757+0000", 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752757+0000", 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103999999999997e-05 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752796+0000", 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8549999999999998e-06 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752802+0000", 2020-11-16T22:33:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0975000000000001e-05 2020-11-16T22:33:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:19.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752813+0000", 2020-11-16T22:33:19.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9260000000000002e-06 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.752817+0000", 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9859999999999999e-05 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:19.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:53330 4.3 e181/137 4:cf4ec11c:::1000000188d.00000017:head v 181'13559, mlcod=181'13556)", 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:17.757659+0000", 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000162666, 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00069478400000000005, 2020-11-16T22:33:19.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757659+0000", 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757659+0000", 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143999999999997e-05 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:19.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757693+0000", 2020-11-16T22:33:19.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2639999999999997e-06 2020-11-16T22:33:19.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:19.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757700+0000", 2020-11-16T22:33:19.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2060000000000007e-06 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757708+0000", 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757712+0000", 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9623999999999999e-05 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:19.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757752+0000", 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5431000000000001e-05 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:17.757777+0000", 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118139 2020-11-16T22:33:19.841 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 10 2020-11-16T22:33:19.842 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:19.843 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:33:19.844 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:19.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~117s 2020-11-16T22:33:19.846 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:19.846 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:19.859+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:20.051 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:20.066+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:20.151 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.166+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.203 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:20.204 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:20.204 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:20.214 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:20.226 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:20.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:20.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:20.257 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:20.273+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:20.353 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.367+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.363 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:20.363 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:20.373 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:20.453 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.468+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.517 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:20.517 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:20.532 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:20.573 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.588+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.656 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:20.671+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:20.712 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:20.713 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:20.722 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:20.765 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.780+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.858 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:20.872+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:20.876 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:20.876 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:20.888 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:20.958 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:20.973+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:21.051 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.052 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.060 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.074+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.062 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:21.159 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:33:21.161 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.175+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.244 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.244 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.254 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:21.261 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.276+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.362 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.377+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.405 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.405 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.415 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:21.463 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.478+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.563 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:21.578+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:21.568 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.568 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.580 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:21.664 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:21.679+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:21.760 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.760 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.764 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:21.779+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:21.771 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:21.865 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:21.880+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:21.960 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:21.960 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:21.965 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:21.980+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:21.971 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:22.066 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:22.080+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:22.135 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:22.135 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:22.146 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:22.187 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:22.202+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:22.230 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:22.268 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:22.282+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:22.341 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:22.341 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:22.351 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:22.368 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:22.383+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:22.379 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:22.403 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:22.403 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:22.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:22.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:22.403 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:22.404 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:22.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:22.405 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:22.405 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:33:22.406 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.8369944372171847 2020-11-16T22:33:22.406 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.8369944372171847 2020-11-16T22:33:22.469 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:22.483+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:22.491 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:22.491 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:22.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 204.50464055800001, 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:22.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:22.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:22.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 203.29649039099999, 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:22.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:22.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:22.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:22.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:22.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:22.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:22.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:22.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.922 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:22.922 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:22.922 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 172.215774536, 2020-11-16T22:33:22.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:22.922 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.923 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:22.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:22.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:22.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:22.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:22.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:22.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:22.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:22.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:22.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:22.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:22.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:23.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:23.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 172.215679096, 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:23.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:23.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:23.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:23.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:23.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:23.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:23.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:23.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 172.21314476399999, 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:23.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:33:23.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:33:23.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:33:23.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:33:23.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:33:23.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18637 4.1 4:8c3af0c3:::1000000188a.00000000:head [trimtrunc 259@3946217] snapc 1=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 171.86272497100001, 2020-11-16T22:33:23.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997, 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:23.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.641983+0000", 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2955e-05 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642006+0000", 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5800000000000002e-06 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642011+0000", 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2020-11-16T22:33:23.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642019+0000", 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642022+0000", 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5204999999999999e-05 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.642057+0000", 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040318623999999997 2020-11-16T22:33:23.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682376+0000", 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9668999999999998e-05 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.682466+0000", 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016198759999999999 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.684085+0000", 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001439308 2020-11-16T22:33:23.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685525+0000", 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5890999999999998e-05 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685571+0000", 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7830000000000002e-05 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.685608+0000", 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043625025999999997 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 169.16033013399999, 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:33:23.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:33:23.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:33:23.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:23586 3.0 3:17051619:::200.0000000f:head [write 4090183~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e139)", 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 142.419819254, 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998, 2020-11-16T22:33:23.786 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084889+0000", 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5345999999999997e-05 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084925+0000", 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999998e-06 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084932+0000", 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1917e-05 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084944+0000", 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7219999999999996e-06 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084950+0000", 2020-11-16T22:33:23.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7470999999999998e-05 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.084977+0000", 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5273999999999999e-05 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085042+0000", 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8197999999999997e-05 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.085141+0000", 2020-11-16T22:33:23.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003107814 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.088248+0000", 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049439332000000002 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137688+0000", 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9957999999999999e-05 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137728+0000", 2020-11-16T22:33:23.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6922000000000001e-05 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:00.137755+0000", 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052865403999999998 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 139.96087279599999, 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:23.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:23.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24065 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1144@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 138.27880181500001, 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002, 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225907+0000", 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3964999999999998e-05 2020-11-16T22:33:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225931+0000", 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225934+0000", 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8589999999999999e-06 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225941+0000", 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.756e-06 2020-11-16T22:33:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225944+0000", 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3402000000000003e-05 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.225977+0000", 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8405000000000005e-05 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226066+0000", 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012572700000000001 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226192+0000", 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021695859999999998 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.228361+0000", 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044528005000000002 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272889+0000", 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1922999999999998e-05 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272931+0000", 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1137999999999998e-05 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.272972+0000", 2020-11-16T22:33:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047065555000000002 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:39633 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2298819~22137] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 138.27835594199999, 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999, 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 39633 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226352+0000", 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0832e-05 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226373+0000", 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3540000000000002e-06 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226378+0000", 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0949999999999998e-06 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226385+0000", 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2270000000000002e-06 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.226388+0000", 2020-11-16T22:33:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013708730000000001 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227759+0000", 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2016999999999999e-05 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.227821+0000", 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046275832000000003 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.274097+0000", 2020-11-16T22:33:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047744229999999999 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24278 4.5 4:a611fa68:::1000000188a.00000002:head [trimtrunc 1181@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e142)", 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 137.367158883, 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001, 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137550+0000", 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4893e-05 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137574+0000", 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0779999999999997e-06 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137579+0000", 2020-11-16T22:33:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0709999999999995e-06 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137587+0000", 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8870000000000001e-06 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137589+0000", 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6246000000000002e-05 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137656+0000", 2020-11-16T22:33:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4323000000000001e-05 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137720+0000", 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8172999999999998e-05 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.137808+0000", 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001628811 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.139437+0000", 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193803000000003 2020-11-16T22:33:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182631+0000", 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4620000000000003e-05 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182675+0000", 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6527999999999998e-05 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:05.182732+0000", 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045182433000000001 2020-11-16T22:33:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:44610 4.0 4:0caefcac:::1000000188a.00000001:head [zero 2893720~29462] snapc 1=[] ondisk+write+known_if_redirected e144)", 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 125.58670106300001, 2020-11-16T22:33:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036, 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 44610 2020-11-16T22:33:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918007+0000", 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9911999999999998e-05 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918027+0000", 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6550000000000003e-06 2020-11-16T22:33:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918032+0000", 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2560000000000002e-06 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918040+0000", 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.318e-06 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918045+0000", 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8257000000000001e-05 2020-11-16T22:33:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918103+0000", 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9557e-05 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.918152+0000", 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044349198999999999 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962502+0000", 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3315000000000001e-05 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962545+0000", 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2567e-05 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:16.962597+0000", 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044590036 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.480062143, 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:40504 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 389@2510211] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.832161595000002, 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998, 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672547+0000", 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3415000000000003e-05 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672600+0000", 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8559999999999996e-06 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672609+0000", 2020-11-16T22:33:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5803000000000001e-05 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672625+0000", 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0939999999999996e-06 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672631+0000", 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4663999999999998e-05 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672676+0000", 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111723 2020-11-16T22:33:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672787+0000", 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015558 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.672943+0000", 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063655059999999999 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.679308+0000", 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.034442592000000001 2020-11-16T22:33:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713751+0000", 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7538000000000002e-05 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713829+0000", 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2500999999999998e-05 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713881+0000", 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041334271999999998 2020-11-16T22:33:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:62889 4.2 4:5e663dd9:::1000000188d.00000002:head [write 2580480~40960 [389@4194304]] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.831107492000001, 2020-11-16T22:33:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477, 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 62889 2020-11-16T22:33:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673601+0000", 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7012000000000001e-05 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673678+0000", 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.464e-06 2020-11-16T22:33:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673685+0000", 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2585999999999999e-05 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673698+0000", 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1329999999999998e-06 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673703+0000", 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180645 2020-11-16T22:33:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673884+0000", 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1970999999999993e-05 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.673946+0000", 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103269 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.674049+0000", 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035807249999999999 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.677630+0000", 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036176901999999997 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713807+0000", 2020-11-16T22:33:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1492e-05 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713868+0000", 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129278 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:33.713997+0000", 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040396477 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.540630041, 2020-11-16T22:33:24.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:24.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:24.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:24.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:24.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:24.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:24.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:72101 4.5 4:ab64ebfc:::1000000188d.00000009:head [write 1708032~23961 [1134@0]] snapc 1=[] ondisk+write+known_if_redirected e180)", 2020-11-16T22:33:24.089 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:13.031218+0000", 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.4734902390000002, 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043574660000000001, 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:24.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 72101 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031218+0000", 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:24.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031218+0000", 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5306999999999998e-05 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:24.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031263+0000", 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5540000000000002e-06 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031267+0000", 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031273+0000", 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.604e-06 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031276+0000", 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3973999999999997e-05 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031310+0000", 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2726999999999996e-05 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031393+0000", 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013983600000000001 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.031532+0000", 2020-11-16T22:33:24.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043099602000000001 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.074632+0000", 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3314999999999997e-05 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.074725+0000", 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9289000000000003e-05 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.074765+0000", 2020-11-16T22:33:24.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.815e-05 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.074793+0000", 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043574660000000001 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:24.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:51892 3.5 3:bec9163d:::200.00000015:head [write 1921597~1923 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e180)", 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:13.036114+0000", 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.4685940960000003, 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041287599000000001, 2020-11-16T22:33:24.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036114+0000", 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036114+0000", 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036161+0000", 2020-11-16T22:33:24.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5419999999999996e-06 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036167+0000", 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1409999999999994e-06 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036176+0000", 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3429999999999998e-06 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036180+0000", 2020-11-16T22:33:24.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8349000000000002e-05 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036218+0000", 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2454000000000006e-05 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.036311+0000", 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040982764999999997 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.077294+0000", 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1165999999999998e-05 2020-11-16T22:33:24.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.077345+0000", 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7080000000000002e-05 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:13.077402+0000", 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041287599000000001 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.108 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53722 3.5 3:bec9163d:::200.00000015:head [write 2911097~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.3179096229999998, 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061281549999999997, 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100505 2020-11-16T22:33:24.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186899+0000", 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5376e-05 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186915+0000", 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6409000000000001e-05 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186941+0000", 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:33:24.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186952+0000", 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119407 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:24.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.187072+0000", 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.05766988 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.244741+0000", 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032219169999999999 2020-11-16T22:33:24.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.247963+0000", 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3133000000000001e-05 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:24.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.248017+0000", 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3825999999999995e-05 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.248080+0000", 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061281549999999997 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.116 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:24.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:24.117 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:24.117 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:24.118 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:33:24.119 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 0.836994 (d645) 2020-11-16T22:33:24.120 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:24.133+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:24.120 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:24.219 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:24.233+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:24.257 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:33:24.258 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:24.266 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:24.320 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:24.335+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:24.390 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:24.390 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:24.390 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:24.390 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:24.391 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:24.399 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:24.405 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:24.421 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:24.436+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:24.522 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:24.537+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:24.525 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:24.525 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:24.525 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:24.525 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:24.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 319.21795531800001, 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.526 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:24.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:24.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:24.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:24.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:24.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:24.534 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 314.21745242999998, 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:33:24.535 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.536 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:24.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:33:24.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:33:24.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:33:24.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:24.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:33:24.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:24.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:33:24.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:33:24.950 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:24.950 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:24.951 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 263.92536081399999, 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:24.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:24.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:24.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:24.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:24.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:24.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:24.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.158 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:25.158 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:25.158 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:25.159 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:25.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:25.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:25.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:25.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:25.367 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 205.62858574000001, 2020-11-16T22:33:25.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:25.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:25.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:25.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:25.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:25.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:25.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:25.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.577 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:25.578 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.579 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:25.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:25.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:25.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 205.33020868099999, 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:25.784 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.785 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:25.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:25.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:25.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:25.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:25.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:25.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:25.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:25.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.992 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:25.993 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:25.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:25.995 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:25.995 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:25.995 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:26.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:26.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:26.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 144.453260667, 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:26.202 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:26.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:26.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:26.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:26.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:26.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:26.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:26.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:26.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:26.819 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 139.400662681, 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:26.820 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:26.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:26.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:26.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:26.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:26.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:27.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:27.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:27.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:27.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:27.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:27.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.031 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:27.031 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.234 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:27.234 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:27.234 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 139.399619355, 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:27.235 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:27.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:27.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:27.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:27.242 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.866023054000003, 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:27.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:27.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:27.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:33:27.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:33:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:33:27.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:27.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:33:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:33:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:27.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:33:27.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:33:27.658 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.865456025999997, 2020-11-16T22:33:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:33:27.659 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:27.862 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:27.862 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:27.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:27.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:27.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:27.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:27.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:27.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:33:27.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:33:28.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:33:28.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.072 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.741425688, 2020-11-16T22:33:28.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:33:28.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:28.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:33:28.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:33:28.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:33:28.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:28.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:33:28.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:28.283 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7928231130000001, 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:28.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:28.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:33:28.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:33:28.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:33:28.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:28.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:33:28.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:33:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:28.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:33:28.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:33:28.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.792730567, 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:28.700 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:28.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:33:28.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:33:28.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:33:28.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:33:28.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:28.912 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7923254669999999, 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:28.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:28.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:33:28.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:28.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:28.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:33:29.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:33:29.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.118 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:33:29.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:33:29.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.123 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.123 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:33:29.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:33:29.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.326 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7922415030000001, 2020-11-16T22:33:29.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:33:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:33:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:33:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7898316249999999, 2020-11-16T22:33:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:29.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:33:29.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:33:29.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:33:29.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:29.343 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7871053240000001, 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:29.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:33:29.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:33:29.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:33:29.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:29.351 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.785431502, 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:29.352 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:33:29.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:33:29.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:33:29.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.361 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.7846911839999999, 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:33:29.362 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:33:29.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.369 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 1.349684028, 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:33:29.370 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:29.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:29.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:29.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:29.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.377 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.378 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:29.378 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:29.378 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:29.379 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:29.379 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:33:29.379 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2020-11-16T22:33:29.381 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:29.381 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:29.382 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:29.382 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:29.382 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:29.382 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:29.382 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:29.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:29.383 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:33:29.383 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:29.394+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:29.384 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:29.480 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:29.494+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:29.530 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:29.530 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:29.541 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:29.580 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:29.595+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:29.682 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:29.696+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:29.725 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:29.725 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 185, 2020-11-16T22:33:29.728 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "2963e9ac-8003-490b-a7b3-aee3b9109ae1", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "created": "2020-11-16T22:23:04.049027+0000", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2020-11-16T22:33:24.949180+0000", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "last_up_change": "2020-11-16T22:31:03.095061+0000", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "last_in_change": "2020-11-16T22:23:09.075128+0000", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_num": 5801984, 2020-11-16T22:33:29.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_set": [ 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "noscrub", 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "pglog_hardlimit", 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "purged_snapdirs", 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_deletes", 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "sortbitwise" 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:29.730 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_version": 4, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "full_ratio": 0.94999998807907104, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_snapshot": "", 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_max": 4, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd": 3, 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "require_min_compat_client": "luminous", 2020-11-16T22:33:29.731 INFO:teuthology.orchestra.run.smithi138.stdout: "min_compat_client": "jewel", 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "require_osd_release": "pacific", 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "pools": [ 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 1, 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "device_health_metrics", 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:07.667948+0000", 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:29.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:29.733 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 1, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 1, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 1, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 1, 2020-11-16T22:33:29.734 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 1, 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "10", 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:29.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:29.738 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:29.739 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:29.740 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:29.740 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:29.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:29.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:29.942 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:29.942 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:29.942 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:29.942 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 1 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:29.943 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_devicehealth": {} 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 2, 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "rbd", 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:12.436040+0000", 2020-11-16T22:33:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:29.945 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:29.946 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:29.947 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "13", 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:30.155 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd": {} 2020-11-16T22:33:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 3, 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_metadata", 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:15.658351+0000", 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "16", 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "options": { 2020-11-16T22:33:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_bias": 4, 2020-11-16T22:33:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_min": 16, 2020-11-16T22:33:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_priority": 5 2020-11-16T22:33:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": "cephfs" 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": 4, 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": "cephfs_data", 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "create_time": "2020-11-16T22:23:16.461191+0000", 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flags_names": "hashpspool", 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type": 1, 2020-11-16T22:33:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 2, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": 1, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_rule": 0, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_count": 0, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_target": 0, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_barrier": 0, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "object_hash": 2, 2020-11-16T22:33:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscale_mode": "off", 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": 8, 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num": 8, 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_placement_num_target": 8, 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_target": 8, 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num_pending": 8, 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "last_pg_merge_meta": { 2020-11-16T22:33:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "source_pgid": "0.0", 2020-11-16T22:33:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "ready_epoch": 0, 2020-11-16T22:33:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_started": 0, 2020-11-16T22:33:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "last_epoch_clean": 0, 2020-11-16T22:33:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "source_version": "0'0", 2020-11-16T22:33:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "target_version": "0'0" 2020-11-16T22:33:30.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.982 INFO:teuthology.orchestra.run.smithi138.stdout: "last_change": "17", 2020-11-16T22:33:30.982 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend": "0", 2020-11-16T22:33:30.982 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "last_force_op_resend_preluminous": "0", 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "auid": 0, 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_mode": "selfmanaged", 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_seq": 0, 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_epoch": 0, 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_snaps": [], 2020-11-16T22:33:30.983 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps": "[]", 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_bytes": 0, 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "quota_max_objects": 0, 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "tiers": [], 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_of": -1, 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "read_tier": -1, 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "write_tier": -1, 2020-11-16T22:33:30.984 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_mode": "none", 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_bytes": 0, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "target_max_objects": 0, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_flush_age": 0, 2020-11-16T22:33:30.985 INFO:teuthology.orchestra.run.smithi138.stdout: "cache_min_evict_age": 0, 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profile": "", 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_params": { 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "none" 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_period": 0, 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_count": 0, 2020-11-16T22:33:30.986 INFO:teuthology.orchestra.run.smithi138.stdout: "use_gmt_hitset": true, 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "min_read_recency_for_promote": 0, 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "min_write_recency_for_promote": 0, 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_grade_decay_rate": 0, 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "hit_set_search_last_n": 0, 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "grade_table": [], 2020-11-16T22:33:30.987 INFO:teuthology.orchestra.run.smithi138.stdout: "stripe_width": 0, 2020-11-16T22:33:31.190 INFO:teuthology.orchestra.run.smithi138.stdout: "expected_num_objects": 0, 2020-11-16T22:33:31.190 INFO:teuthology.orchestra.run.smithi138.stdout: "fast_read": false, 2020-11-16T22:33:31.190 INFO:teuthology.orchestra.run.smithi138.stdout: "options": {}, 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: "application_metadata": { 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: "cephfs": { 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: "data": "cephfs" 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "d7ac8050-ab58-4ba1-8b67-54b1751e09c0", 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 0, 2020-11-16T22:33:31.192 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 0.8369903564453125, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 119, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 141, 2020-11-16T22:33:31.193 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6800", 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6801", 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.196 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:31.196 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6802", 2020-11-16T22:33:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6803", 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6806", 2020-11-16T22:33:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6807", 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.401 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6804", 2020-11-16T22:33:31.402 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.173:6805", 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20752 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.403 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.404 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.173:6801/20752", 2020-11-16T22:33:31.606 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.173:6803/20752", 2020-11-16T22:33:31.606 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.173:6807/20752", 2020-11-16T22:33:31.606 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.173:6805/20752", 2020-11-16T22:33:31.606 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: "exists" 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "94f7af5a-9a19-4ee0-8078-95c6902e33f2", 2020-11-16T22:33:31.607 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 0.463592529296875, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 0.7154998779296875, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 0, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 0, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 8, 2020-11-16T22:33:31.608 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 183, 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 0, 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.609 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6800", 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6801", 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.610 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6802", 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6803", 2020-11-16T22:33:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6806", 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6807", 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6804", 2020-11-16T22:33:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.194:6805", 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 20794 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.194:6801/20794", 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.194:6803/20794", 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.194:6807/20794", 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.194:6805/20794", 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:33:31.819 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: "uuid": "92637938-4ff7-497e-b599-40612e2d47dd", 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2020-11-16T22:33:31.820 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "weight": 1, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_affinity": 1, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_begin": 63, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "last_clean_end": 69, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "up_from": 108, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "up_thru": 184, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "down_at": 70, 2020-11-16T22:33:31.821 INFO:teuthology.orchestra.run.smithi138.stdout: "lost_at": 0, 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addrs": { 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6800", 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6801", 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:31.823 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addrs": { 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6802", 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:31.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6803", 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.026 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addrs": { 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6806", 2020-11-16T22:33:32.027 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6807", 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addrs": { 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6804", 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.122:6805", 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2959 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.030 INFO:teuthology.orchestra.run.smithi138.stdout: "public_addr": "172.21.15.122:6801/2959", 2020-11-16T22:33:32.031 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_addr": "172.21.15.122:6803/2959", 2020-11-16T22:33:32.031 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_back_addr": "172.21.15.122:6807/2959", 2020-11-16T22:33:32.031 INFO:teuthology.orchestra.run.smithi138.stdout: "heartbeat_front_addr": "172.21.15.122:6805/2959", 2020-11-16T22:33:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "state": [ 2020-11-16T22:33:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "exists", 2020-11-16T22:33:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "up" 2020-11-16T22:33:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_xinfo": [ 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 0, 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.886258+0000", 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 141 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 1, 2020-11-16T22:33:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.939013+0000", 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 0 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": 2, 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "down_stamp": "0.000000", 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_probability": 0, 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "laggy_interval": 0, 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "features": 4540138297136906239, 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "old_weight": 0, 2020-11-16T22:33:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "last_purged_snaps_scrub": "2020-11-16T22:23:07.955568+0000", 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "dead_epoch": 70 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap": [], 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_upmap_items": [], 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_temp": [ 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "1.0", 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.0", 2020-11-16T22:33:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.3", 2020-11-16T22:33:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.5", 2020-11-16T22:33:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.6", 2020-11-16T22:33:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "2.7", 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.244 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.4", 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.245 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "3.6", 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.1", 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.448 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.2", 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2020-11-16T22:33:32.449 INFO:teuthology.orchestra.run.smithi138.stdout: "pgid": "4.4", 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "osds": [ 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: 2, 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: 1 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "primary_temp": [], 2020-11-16T22:33:32.450 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklist": {}, 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "erasure_code_profiles": { 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "default": { 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-failure-domain": "osd", 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "k": "2", 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "m": "1", 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "plugin": "jerasure", 2020-11-16T22:33:32.451 INFO:teuthology.orchestra.run.smithi138.stdout: "ruleset-failure-domain": "osd", 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "technique": "reed_sol_van" 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "removed_snaps_queue": [], 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "new_removed_snaps": [], 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "new_purged_snaps": [], 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_node_flags": {}, 2020-11-16T22:33:32.452 INFO:teuthology.orchestra.run.smithi138.stdout: "device_class_flags": {}, 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode": { 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_enabled": false, 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_bucket_count": 0, 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "degraded_stretch_mode": 0, 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_stretch_mode": 0, 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: "stretch_mode_bucket": 0 2020-11-16T22:33:32.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2020-11-16T22:33:32.454 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:33:32.454 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:32.455 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:33:32.457 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:32.458 INFO:tasks.thrashosds.thrasher:j is {'epoch': 185, 'fsid': '2963e9ac-8003-490b-a7b3-aee3b9109ae1', 'created': '2020-11-16T22:23:04.049027+0000', 'modified': '2020-11-16T22:33:24.949180+0000', 'last_up_change': '2020-11-16T22:31:03.095061+0000', 'last_in_change': '2020-11-16T22:23:09.075128+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:23:07.667948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:23:12.436040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:23:15.658351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:23:16.461191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}], 'osds': [{'osd': 0, 'uuid': 'd7ac8050-ab58-4ba1-8b67-54b1751e09c0', 'up': 0, 'in': 1, 'weight': 0.8369903564453125, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 119, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 20752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 20752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 20752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 20752}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 20752}]}, 'public_addr': '172.21.15.173:6801/20752', 'cluster_addr': '172.21.15.173:6803/20752', 'heartbeat_back_addr': '172.21.15.173:6807/20752', 'heartbeat_front_addr': '172.21.15.173:6805/20752', 'state': ['exists']}, {'osd': 1, 'uuid': '94f7af5a-9a19-4ee0-8078-95c6902e33f2', 'up': 1, 'in': 1, 'weight': 0.463592529296875, 'primary_affinity': 0.7154998779296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 183, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6800', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6801', 'nonce': 20794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6802', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6803', 'nonce': 20794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6806', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6807', 'nonce': 20794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6804', 'nonce': 20794}, {'type': 'v1', 'addr': '172.21.15.194:6805', 'nonce': 20794}]}, 'public_addr': '172.21.15.194:6801/20794', 'cluster_addr': '172.21.15.194:6803/20794', 'heartbeat_back_addr': '172.21.15.194:6807/20794', 'heartbeat_front_addr': '172.21.15.194:6805/20794', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '92637938-4ff7-497e-b599-40612e2d47dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 63, 'last_clean_end': 69, 'up_from': 108, 'up_thru': 184, 'down_at': 70, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 2959}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 2959}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 2959}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 2959}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 2959}]}, 'public_addr': '172.21.15.122:6801/2959', 'cluster_addr': '172.21.15.122:6803/2959', 'heartbeat_back_addr': '172.21.15.122:6807/2959', 'heartbeat_front_addr': '172.21.15.122:6805/2959', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.886258+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.939013+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-11-16T22:23:07.955568+0000', 'dead_epoch': 70}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '1.0', 'osds': [2, 1]}, {'pgid': '2.0', 'osds': [2, 1]}, {'pgid': '2.3', 'osds': [2, 1]}, {'pgid': '2.5', 'osds': [2, 1]}, {'pgid': '2.6', 'osds': [2, 1]}, {'pgid': '2.7', 'osds': [2, 1]}, {'pgid': '3.4', 'osds': [2, 1]}, {'pgid': '3.6', 'osds': [2, 1]}, {'pgid': '4.1', 'osds': [2, 1]}, {'pgid': '4.2', 'osds': [2, 1]}, {'pgid': '4.4', 'osds': [2, 1]}], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2020-11-16T22:33:32.458 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:33:32.459 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:32.469+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:32.555 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:32.570+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:32.609 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:32.609 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:32.620 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:32.654 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:32.660 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:32.675+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:32.678 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:32.681 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:32.681 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:32.757 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:32.772+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:32.763 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:32.763 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:32.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.763 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:57009 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1484@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e185)", 2020-11-16T22:33:32.763 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:32.777671+0000", 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.000391405, 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00041858299999999999, 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:32.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777671+0000", 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777671+0000", 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4666000000000001e-05 2020-11-16T22:33:32.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777695+0000", 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0699999999999997e-06 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:32.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777700+0000", 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0288999999999999e-05 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777711+0000", 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3900000000000001e-06 2020-11-16T22:33:32.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777716+0000", 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2404999999999998e-05 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:32.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777758+0000", 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000103424 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777862+0000", 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00011339 2020-11-16T22:33:32.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777975+0000", 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00030463400000000002 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:32.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:57011 4.3 4.388372f3 (undecoded) ondisk+write+known_if_redirected+full_force e185)", 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:32.777778+0000", 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00028411899999999999, 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00039400999999999998, 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:32.771 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "queued for pg", 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777778+0000", 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777778+0000", 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6351999999999999e-05 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777834+0000", 2020-11-16T22:33:32.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0625000000000001e-05 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777845+0000", 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5483999999999999e-05 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777860+0000", 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5670000000000002e-06 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:32.777866+0000", 2020-11-16T22:33:32.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8028000000000005e-05 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:33:32.776 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:32.777 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:32.823 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:33:32.824 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:33:32.835 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2020-11-16T22:33:31.759441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":144663213,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":472,"num_objects_misplaced":1376,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":45369,"num_read_kb":1540515,"num_write":90852,"num_write_kb":864877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6804,"num_bytes_recovered":139054382,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79944,"ondisk_log_size":79944,"up":31,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":691096,"kb_used_data":203604,"kb_used_omap":26,"kb_used_meta":487205,"kb_avail":282424424,"statfs":{"total":289910292480,"available":289202610176,"internally_reserved":0,"allocated":208490496,"data_stored":249457804,"data_compressed":19860253,"data_compressed_allocated":68411392,"data_compressed_original":136810190,"omap_allocated":27325,"internal_metadata":498898243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9233946,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-880,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":692,"num_read_kb":19373,"num_write":1349,"num_write_kb":9598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001669"},"pg_stats":[{"pgid":"4.2","version":"185'1955","reported_seq":"2834","reported_epoch":"185","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:27.105999+0000","last_change":"2020-11-16T22:32:04.140287+0000","last_active":"2020-11-16T22:33:27.105999+0000","last_peered":"2020-11-16T22:33:27.105999+0000","last_clean":"2020-11-16T22:33:27.105999+0000","last_became_active":"2020-11-16T22:32:04.136650+0000","last_became_peered":"2020-11-16T22:32:04.136650+0000","last_unstale":"2020-11-16T22:33:27.105999+0000","last_undegraded":"2020-11-16T22:33:27.105999+0000","last_fullsized":"2020-11-16T22:33:27.105999+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":1955,"ondisk_log_size":1955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6711215,"num_objects":457,"num_object_clones":0,"num_object_copies":914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":457,"num_objects_unfound":0,"num_objects_dirty":457,"num_whiteouts":0,"num_read":730,"num_read_kb":30691,"num_write":2074,"num_write_kb":19958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"51","reported_epoch":"184","state":"active+undersized","last_fresh":"2020-11-16T22:33:23.950737+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:33:23.950737+0000","last_peered":"2020-11-16T22:33:23.950737+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:33:23.950737+0000","last_undegraded":"2020-11-16T22:33:23.950737+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"185'5172","reported_seq":"5334","reported_epoch":"185","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:27.148176+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:33:27.148176+0000","last_peered":"2020-11-16T22:33:27.148176+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:33:27.148176+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5172,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16411072,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5169,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"185'5013","reported_seq":"10305","reported_epoch":"185","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:27.162252+0000","last_change":"2020-11-16T22:32:38.960533+0000","last_active":"2020-11-16T22:33:27.162252+0000","last_peered":"2020-11-16T22:33:27.162252+0000","last_clean":"2020-11-16T22:33:27.162252+0000","last_became_active":"2020-11-16T22:32:38.951761+0000","last_became_peered":"2020-11-16T22:32:38.951761+0000","last_unstale":"2020-11-16T22:33:27.162252+0000","last_undegraded":"2020-11-16T22:33:27.162252+0000","last_fullsized":"2020-11-16T22:33:27.162252+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":5013,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10237053,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":447,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":2440,"num_read_kb":81344,"num_write":5031,"num_write_kb":33753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"114","reported_epoch":"184","state":"active+undersized","last_fresh":"2020-11-16T22:33:23.949513+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:33:23.949513+0000","last_peered":"2020-11-16T22:33:23.949513+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:33:23.949513+0000","last_undegraded":"2020-11-16T22:33:23.949513+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"168'4360","reported_seq":"4524","reported_epoch":"184","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:23.949925+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:33:23.949925+0000","last_peered":"2020-11-16T22:33:23.949925+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:33:23.949925+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4360,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":16951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"185'2538","reported_seq":"3708","reported_epoch":"185","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:27.121219+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:33:27.121219+0000","last_peered":"2020-11-16T22:33:27.121219+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:33:27.121219+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":2538,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5886755,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":970,"num_read_kb":36272,"num_write":2627,"num_write_kb":16669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":451}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"51","reported_epoch":"184","state":"active+undersized","last_fresh":"2020-11-16T22:33:23.950565+0000","last_change":"2020-11-16T22:31:04.115597+0000","last_active":"2020-11-16T22:33:23.950565+0000","last_peered":"2020-11-16T22:33:23.950565+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115597+0000","last_became_peered":"2020-11-16T22:31:04.115597+0000","last_unstale":"2020-11-16T22:33:23.950565+0000","last_undegraded":"2020-11-16T22:33:23.950565+0000","last_fullsized":"2020-11-16T22:31:04.110788+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"165'10812","reported_seq":"11011","reported_epoch":"184","state":"active+clean","last_fresh":"2020-11-16T22:33:23.949418+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:33:23.949418+0000","last_peered":"2020-11-16T22:33:23.949418+0000","last_clean":"2020-11-16T22:33:23.949418+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:33:23.949418+0000","last_undegraded":"2020-11-16T22:33:23.949418+0000","last_fullsized":"2020-11-16T22:33:23.949418+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10012,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"185'10545","reported_seq":"14986","reported_epoch":"185","state":"active+clean","last_fresh":"2020-11-16T22:33:27.169891+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:33:27.169891+0000","last_peered":"2020-11-16T22:33:27.169891+0000","last_clean":"2020-11-16T22:33:27.169891+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:33:27.169891+0000","last_undegraded":"2020-11-16T22:33:27.169891+0000","last_fullsized":"2020-11-16T22:33:27.169891+0000","mapping_epoch":117,"log_start":"88'500","ondisk_log_start":"88'500","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10045,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2924420,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":4273,"num_read_kb":101851,"num_write":10576,"num_write_kb":92622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"136","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949598+0000","last_change":"2020-11-16T22:32:02.128340+0000","last_active":"2020-11-16T22:33:23.949598+0000","last_peered":"2020-11-16T22:33:23.949598+0000","last_clean":"2020-11-16T22:33:23.949598+0000","last_became_active":"2020-11-16T22:32:02.127967+0000","last_became_peered":"2020-11-16T22:32:02.127967+0000","last_unstale":"2020-11-16T22:33:23.949598+0000","last_undegraded":"2020-11-16T22:33:23.949598+0000","last_fullsized":"2020-11-16T22:33:23.949598+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"122","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949109+0000","last_change":"2020-11-16T22:32:39.953425+0000","last_active":"2020-11-16T22:33:23.949109+0000","last_peered":"2020-11-16T22:33:23.949109+0000","last_clean":"2020-11-16T22:33:23.949109+0000","last_became_active":"2020-11-16T22:32:39.952851+0000","last_became_peered":"2020-11-16T22:32:39.952851+0000","last_unstale":"2020-11-16T22:33:23.949109+0000","last_undegraded":"2020-11-16T22:33:23.949109+0000","last_fullsized":"2020-11-16T22:33:23.949109+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"165'4341","reported_seq":"4497","reported_epoch":"184","state":"active+clean","last_fresh":"2020-11-16T22:33:23.963143+0000","last_change":"2020-11-16T22:33:23.958874+0000","last_active":"2020-11-16T22:33:23.963143+0000","last_peered":"2020-11-16T22:33:23.963143+0000","last_clean":"2020-11-16T22:33:23.963143+0000","last_became_active":"2020-11-16T22:33:23.957142+0000","last_became_peered":"2020-11-16T22:33:23.957142+0000","last_unstale":"2020-11-16T22:33:23.963143+0000","last_undegraded":"2020-11-16T22:33:23.963143+0000","last_fullsized":"2020-11-16T22:33:23.963143+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4341,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":12835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"185'2617","reported_seq":"5189","reported_epoch":"185","state":"active+clean","last_fresh":"2020-11-16T22:33:27.051800+0000","last_change":"2020-11-16T22:33:24.965873+0000","last_active":"2020-11-16T22:33:27.051800+0000","last_peered":"2020-11-16T22:33:27.051800+0000","last_clean":"2020-11-16T22:33:27.051800+0000","last_became_active":"2020-11-16T22:33:24.959880+0000","last_became_peered":"2020-11-16T22:33:24.959880+0000","last_unstale":"2020-11-16T22:33:27.051800+0000","last_undegraded":"2020-11-16T22:33:27.051800+0000","last_fullsized":"2020-11-16T22:33:27.051800+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":2617,"ondisk_log_size":2617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1912281,"num_objects":438,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":1064,"num_read_kb":31108,"num_write":2680,"num_write_kb":16279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"118","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949143+0000","last_change":"2020-11-16T22:32:38.951861+0000","last_active":"2020-11-16T22:33:23.949143+0000","last_peered":"2020-11-16T22:33:23.949143+0000","last_clean":"2020-11-16T22:33:23.949143+0000","last_became_active":"2020-11-16T22:32:38.951148+0000","last_became_peered":"2020-11-16T22:32:38.951148+0000","last_unstale":"2020-11-16T22:33:23.949143+0000","last_undegraded":"2020-11-16T22:33:23.949143+0000","last_fullsized":"2020-11-16T22:33:23.949143+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"90'28","reported_seq":"225","reported_epoch":"184","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:23.949265+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:33:23.949265+0000","last_peered":"2020-11-16T22:33:23.949265+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:33:23.949265+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":217756,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":26,"num_write_kb":231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"185'13879","reported_seq":"30130","reported_epoch":"185","state":"active+clean","last_fresh":"2020-11-16T22:33:29.698896+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:33:29.698896+0000","last_peered":"2020-11-16T22:33:29.698896+0000","last_clean":"2020-11-16T22:33:29.698896+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:33:29.698896+0000","last_undegraded":"2020-11-16T22:33:29.698896+0000","last_fullsized":"2020-11-16T22:33:29.698896+0000","mapping_epoch":137,"log_start":"95'3800","ondisk_log_start":"95'3800","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10079,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15940658,"num_objects":466,"num_object_clones":0,"num_object_copies":932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":466,"num_whiteouts":0,"num_read":15148,"num_read_kb":373394,"num_write":17738,"num_write_kb":303290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"94","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949988+0000","last_change":"2020-11-16T22:32:39.955775+0000","last_active":"2020-11-16T22:33:23.949988+0000","last_peered":"2020-11-16T22:33:23.949988+0000","last_clean":"2020-11-16T22:33:23.949988+0000","last_became_active":"2020-11-16T22:32:39.955217+0000","last_became_peered":"2020-11-16T22:32:39.955217+0000","last_unstale":"2020-11-16T22:33:23.949988+0000","last_undegraded":"2020-11-16T22:33:23.949988+0000","last_fullsized":"2020-11-16T22:33:23.949988+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"181'2238","reported_seq":"2238","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949401+0000","last_change":"2020-11-16T22:32:39.957132+0000","last_active":"2020-11-16T22:33:23.949401+0000","last_peered":"2020-11-16T22:33:23.949401+0000","last_clean":"2020-11-16T22:33:23.949401+0000","last_became_active":"2020-11-16T22:32:39.955410+0000","last_became_peered":"2020-11-16T22:32:39.955410+0000","last_unstale":"2020-11-16T22:33:23.949401+0000","last_undegraded":"2020-11-16T22:33:23.949401+0000","last_fullsized":"2020-11-16T22:33:23.949401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2238,"ondisk_log_size":2238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389251,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2235,"num_write_kb":8514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":6}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"185'10027","reported_seq":"22652","reported_epoch":"185","state":"active+clean","last_fresh":"2020-11-16T22:33:27.162204+0000","last_change":"2020-11-16T22:33:23.961331+0000","last_active":"2020-11-16T22:33:27.162204+0000","last_peered":"2020-11-16T22:33:27.162204+0000","last_clean":"2020-11-16T22:33:27.162204+0000","last_became_active":"2020-11-16T22:33:23.953893+0000","last_became_peered":"2020-11-16T22:33:23.953893+0000","last_unstale":"2020-11-16T22:33:27.162204+0000","last_undegraded":"2020-11-16T22:33:27.162204+0000","last_fullsized":"2020-11-16T22:33:27.162204+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10027,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6873901,"num_objects":439,"num_object_clones":0,"num_object_copies":878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":9692,"num_read_kb":392939,"num_write":10544,"num_write_kb":138905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"144'9","reported_seq":"100","reported_epoch":"184","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:33:23.950918+0000","last_change":"2020-11-16T22:31:04.115488+0000","last_active":"2020-11-16T22:33:23.950918+0000","last_peered":"2020-11-16T22:33:23.950918+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:33:23.950918+0000","last_undegraded":"2020-11-16T22:31:04.111159+0000","last_fullsized":"2020-11-16T22:31:04.111040+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194879,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":5}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"100","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949403+0000","last_change":"2020-11-16T22:32:38.951880+0000","last_active":"2020-11-16T22:33:23.949403+0000","last_peered":"2020-11-16T22:33:23.949403+0000","last_clean":"2020-11-16T22:33:23.949403+0000","last_became_active":"2020-11-16T22:32:38.951168+0000","last_became_peered":"2020-11-16T22:32:38.951168+0000","last_unstale":"2020-11-16T22:33:23.949403+0000","last_undegraded":"2020-11-16T22:33:23.949403+0000","last_fullsized":"2020-11-16T22:33:23.949403+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"185'9235","reported_seq":"20643","reported_epoch":"185","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:27.165034+0000","last_change":"2020-11-16T22:32:39.965767+0000","last_active":"2020-11-16T22:33:27.165034+0000","last_peered":"2020-11-16T22:33:27.165034+0000","last_clean":"2020-11-16T22:33:27.165034+0000","last_became_active":"2020-11-16T22:32:39.955272+0000","last_became_peered":"2020-11-16T22:32:39.955272+0000","last_unstale":"2020-11-16T22:33:27.165034+0000","last_undegraded":"2020-11-16T22:33:27.165034+0000","last_fullsized":"2020-11-16T22:33:27.165034+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9235,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6243682,"num_objects":458,"num_object_clones":0,"num_object_copies":916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":458,"num_objects_unfound":0,"num_objects_dirty":458,"num_whiteouts":0,"num_read":11026,"num_read_kb":492685,"num_write":10354,"num_write_kb":153365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":458}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"118","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949993+0000","last_change":"2020-11-16T22:32:38.951920+0000","last_active":"2020-11-16T22:33:23.949993+0000","last_peered":"2020-11-16T22:33:23.949993+0000","last_clean":"2020-11-16T22:33:23.949993+0000","last_became_active":"2020-11-16T22:32:38.951358+0000","last_became_peered":"2020-11-16T22:32:38.951358+0000","last_unstale":"2020-11-16T22:33:23.949993+0000","last_undegraded":"2020-11-16T22:33:23.949993+0000","last_fullsized":"2020-11-16T22:33:23.949993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"180'2271","reported_seq":"2322","reported_epoch":"184","state":"active+clean+remapped","last_fresh":"2020-11-16T22:33:23.949718+0000","last_change":"2020-11-16T22:32:02.132544+0000","last_active":"2020-11-16T22:33:23.949718+0000","last_peered":"2020-11-16T22:33:23.949718+0000","last_clean":"2020-11-16T22:33:23.949718+0000","last_became_active":"2020-11-16T22:32:02.132057+0000","last_became_peered":"2020-11-16T22:32:02.132057+0000","last_unstale":"2020-11-16T22:33:23.949718+0000","last_undegraded":"2020-11-16T22:33:23.949718+0000","last_fullsized":"2020-11-16T22:33:23.949718+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2271,"ondisk_log_size":2271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2289,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":56729965,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":451,"num_objects_misplaced":1362,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":45343,"num_read_kb":1540284,"num_write":61624,"num_write_kb":774841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76738560,"data_stored":63018304,"data_compressed":5629901,"data_compressed_allocated":10993664,"data_compressed_original":21974734,"omap_allocated":0,"internal_metadata":0},"log_size":51509,"ondisk_log_size":51509,"up":12,"acting":15,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":87933248,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":26,"num_read_kb":231,"num_write":29224,"num_write_kb":90016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":106481772,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127148032,"data_stored":184623978,"data_compressed":14251271,"data_compressed_allocated":57548800,"data_compressed_original":115097600,"omap_allocated":10534,"internal_metadata":0},"log_size":28431,"ondisk_log_size":28431,"up":10,"acting":12,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":1,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":13,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468011,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":264684,"kb_used_data":91236,"kb_used_omap":26,"kb_used_meta":173413,"kb_avail":94107156,"statfs":{"total":96636764160,"available":96365727744,"internally_reserved":0,"allocated":93425664,"data_stored":112725976,"data_compressed":8747113,"data_compressed_allocated":30773248,"data_compressed_original":61546496,"omap_allocated":27325,"internal_metadata":177575235},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:33:00 2020","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.57599999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73799999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.503}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109904,"kb_used_data":35336,"kb_used_omap":0,"kb_used_meta":74560,"kb_avail":94261936,"statfs":{"total":96636764160,"available":96524222464,"internally_reserved":0,"allocated":36184064,"data_stored":55423993,"data_compressed":6804590,"data_compressed_allocated":25083904,"data_compressed_original":50167808,"omap_allocated":0,"internal_metadata":76349440},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:30:20 2020","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.71899999999999997},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":1.028},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316508,"kb_used_data":77032,"kb_used_omap":0,"kb_used_meta":239232,"kb_avail":94055332,"statfs":{"total":96636764160,"available":96312659968,"internally_reserved":0,"allocated":78880768,"data_stored":81307835,"data_compressed":4308550,"data_compressed_allocated":12554240,"data_compressed_original":25095886,"omap_allocated":0,"internal_metadata":244973568},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:33:01 2020","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80600000000000005,"5min":0.998,"15min":0.998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.52400000000000002,"15min":0.53400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":3.4990000000000001}]}]}]}} 2020-11-16T22:33:33.651 INFO:tasks.thrashosds.thrasher:Setting 4.5 to [2, 1] 2020-11-16T22:33:33.652 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.5', '2', '1'] 2020-11-16T22:33:33.652 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.5 2 1 2020-11-16T22:33:33.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~132s 2020-11-16T22:33:33.654 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:33.654 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:33.654 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:33.655 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:33.655 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:33.655 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:33.656 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:33.668+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:33:33.754 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:33.768+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:33.855 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:33.856 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 328.54800574699999, 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:33.857 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:33.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:33.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:33.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:33.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:25 4.2 4:584e1577:::10000000217.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 323.54750285900002, 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182, 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:33.866 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 25 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.321079+0000", 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0094973550000000007 2020-11-16T22:33:33.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330577+0000", 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.9129999999999997e-06 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330584+0000", 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1702000000000001e-05 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330595+0000", 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.314e-06 2020-11-16T22:33:33.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330602+0000", 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00037341499999999999 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:33.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.330975+0000", 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000196906 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.331172+0000", 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.029836769999999999 2020-11-16T22:33:33.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361009+0000", 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0411000000000001e-05 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:33.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361069+0000", 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0395999999999995e-05 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:10.361140+0000", 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040060182 2020-11-16T22:33:33.874 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 273.25541124300003, 2020-11-16T22:33:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:33.876 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.078 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:34.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.080 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.081 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.082 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:34.083 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:34.084 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.085 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 214.95863616899999, 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 214.66025911, 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.097 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.098 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:34.099 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.100 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:34.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:34.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.311 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 153.78331109600001, 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:34.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:34.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:34.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 148.73071311000001, 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:34.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:34.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:34.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:34.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 148.72966978400001, 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:34.530 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:34.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:34.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:34.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:40506 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 389@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e168)", 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 60.196073482999999, 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018, 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.538 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672509+0000", 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2419999999999999e-05 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672531+0000", 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8330000000000004e-06 2020-11-16T22:33:34.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672535+0000", 2020-11-16T22:33:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4849999999999997e-06 2020-11-16T22:33:34.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672541+0000", 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9019999999999999e-06 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.672544+0000", 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.034675309000000001 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707220+0000", 2020-11-16T22:33:34.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00063443099999999997 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707854+0000", 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7540000000000006e-05 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.707942+0000", 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0041681569999999996 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:34.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.712110+0000", 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0012671340000000001 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713377+0000", 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9858999999999999e-05 2020-11-16T22:33:34.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713417+0000", 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2110000000000003e-05 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713449+0000", 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.04094018 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:40504 4.5 e168/117 4:ab64ebfc:::1000000188d.00000009:head v 168'6189, mlcod=168'6187)", 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 60.195506455, 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879, 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.750 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673076+0000", 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0063e-05 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673106+0000", 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7389999999999999e-06 2020-11-16T22:33:34.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673111+0000", 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4980000000000003e-06 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673117+0000", 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.0340000000000001e-06 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673120+0000", 2020-11-16T22:33:34.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3581000000000001e-05 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673174+0000", 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1335000000000002e-05 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.673195+0000", 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040073698999999997 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713269+0000", 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4929999999999998e-05 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:33.713324+0000", 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.040247879 2020-11-16T22:33:34.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:63468 4.4 4:218b21fa:::1000000188d.00000001:head [zero 1489536~62043] snapc 1=[] ondisk+write+known_if_redirected e168)", 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:34.758 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 58.071476117000003, 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001, 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:34.759 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 63468 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797106+0000", 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9086999999999999e-05 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797125+0000", 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4339999999999998e-06 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797135+0000", 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8380000000000002e-06 2020-11-16T22:33:34.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797142+0000", 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4470000000000002e-06 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.797146+0000", 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.038568104999999998 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835714+0000", 2020-11-16T22:33:34.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7158999999999997e-05 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.835801+0000", 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0033729179999999999 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:32:35.839174+0000", 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042067988000000001 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.122873542000001, 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.767 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:33:34.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:33:34.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:33:34.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:33:34.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:34.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:34.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.122780995999999, 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:33:34.979 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:34.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:34.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:33:34.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.983 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:33:34.984 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.985 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:34.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:34.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:33:34.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:33:34.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.190 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:35.190 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.122375895999999, 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:35.191 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:33:35.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:33:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:33:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:33:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.122291932, 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:35.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:35.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:35.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:35.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:33:35.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:33:35.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:33:35.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:35.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:33:35.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:33:35.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:33:35.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:33:35.820 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.119882054, 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:35.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:33:35.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:35.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:35.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:35.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:33:36.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:33:36.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:36.026 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:33:36.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:33:36.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:36.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:33:36.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:33:36.234 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:36.234 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:36.234 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.117155753, 2020-11-16T22:33:36.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:36.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:36.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:33:36.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:33:36.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:33:36.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.115481931, 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:36.447 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:36.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:36.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:36.650 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:36.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:36.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:33:36.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:36.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:33:36.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:33:36.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:33:36.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:36.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:33:36.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:36.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:36.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:33:36.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:33:36.863 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:36.863 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.066 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.114741613, 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:37.067 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.068 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:37.069 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:33:37.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:33:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:33:37.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:37.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:33:37.277 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:37.278 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 10.679734457, 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:37.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:37.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:37.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:33:37.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.487 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:33:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:33:37.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:37.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.692 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:37.693 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:37.693 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:37.693 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:33.869+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:37.694 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:37.695 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:33:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:set 4.5 pg_upmap mapping to [2,1] 2020-11-16T22:33:37.697 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:37.698 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:37.709+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:37.796 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:37.810+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:37.839 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:77805 4.5 4:ae4c5600:::1000000188d.00000012:head [zero 1513271~7096] snapc 1=[] ondisk+write+known_if_redirected e187)", 2020-11-16T22:33:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:37.848038+0000", 2020-11-16T22:33:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0071101840000000003, 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071279819999999997, 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 77805 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848038+0000", 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848038+0000", 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2655000000000001e-05 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848111+0000", 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5400000000000001e-06 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848120+0000", 2020-11-16T22:33:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7280000000000001e-05 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848138+0000", 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0800000000000006e-06 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848146+0000", 2020-11-16T22:33:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7555000000000002e-05 2020-11-16T22:33:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848183+0000", 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8020000000000003e-05 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:37.848251+0000", 2020-11-16T22:33:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021312999999999999 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:33:38.053 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:38.054 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:38.054 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:38.055 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:38.056 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:38.056 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:38.057 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:38.068+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:38.154 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:38.169+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:38.246 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:38.246 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:38.255 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:38.270+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:38.257 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:38.356 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:38.370+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:38.449 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:38.450 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:38.457 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:38.471+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:38.459 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:38.557 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:38.572+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:38.614 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:38.615 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:38.625 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:38.658 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:38.672+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:38.759 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:38.773+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:38.803 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:38.803 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:38.813 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:38.859 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:38.873+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:38.960 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:38.975+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:38.983 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:33:39.010 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.011 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:39.066 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:39.081+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:39.162 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:39.176+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:39.163 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.163 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.173 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:39.262 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:39.277+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:39.359 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.360 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.364 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:39.378+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:39.374 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:39.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~141s 2020-11-16T22:33:39.464 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:39.478+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:39.559 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.559 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.565 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:39.579+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:39.569 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:39.665 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:39.679+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:39.712 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.712 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.724 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:39.766 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:39.780+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:39.867 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:39.881+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:39.923 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:39.923 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:39.934 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:39.968 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:39.982+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.057 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:40.069 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:40.082+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:40.113 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:40.113 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:40.123 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:40.169 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.183+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.205 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:40.227 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:40.227 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:40.227 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:40.228 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:40.228 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:40.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:40.230 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:40.270 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.284+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.274 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:78340 4.4 4:3be3f0ca:::1000000188d.00000007:head [write 982356~15491 [1632@0] in=15491b] snapc 1=[] ondisk+write+known_if_redirected e188)", 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:40.284237+0000", 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0052878450000000002, 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005307361, 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 78340 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284237+0000", 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284237+0000", 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1266000000000004e-05 2020-11-16T22:33:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284318+0000", 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2099999999999996e-06 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284326+0000", 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2182e-05 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284338+0000", 2020-11-16T22:33:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3339999999999998e-06 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284344+0000", 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6899e-05 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284381+0000", 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6311999999999997e-05 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284457+0000", 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010620000000000001 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284564+0000", 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032640300000000001 2020-11-16T22:33:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:78338 4.3 e188/137 4:cf4ec11c:::1000000188d.00000017:head v 188'14225, mlcod=188'14223)", 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:40.284696+0000", 2020-11-16T22:33:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.004829196, 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048975640000000001, 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284696+0000", 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284696+0000", 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014738800000000001 2020-11-16T22:33:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284843+0000", 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0713000000000001e-05 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284854+0000", 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4195999999999999e-05 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284868+0000", 2020-11-16T22:33:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8390000000000001e-06 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284875+0000", 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3665000000000003e-05 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284919+0000", 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1273000000000001e-05 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:40.284940+0000", 2020-11-16T22:33:40.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024407399999999999 2020-11-16T22:33:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:40.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:40.491 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:33:40.491 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:40.492 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:40.493 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.506+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.592 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.607+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.692 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:40.692 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:40.692 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:40.692 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:40.693 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:40.695 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.709+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.707 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:40.795 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.810+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.896 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:40.911+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:40.898 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:40.898 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:40.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:40.898 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:40.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 222.91112598199999, 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:40.899 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:40.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:40.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:40.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:40.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:40.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:40.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 221.702975815, 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:40.909 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:40.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:40.910 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:40.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:40.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:40.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:41.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:41.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.330 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 190.62225996000001, 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:41.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:41.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:41.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:41.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:41.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 190.62216452000001, 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:41.342 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:41.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:41.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:41.352 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:18588 3.0 3:1e8917d7:::200.0000000e:head [write 1826488~1922 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 190.619630188, 2020-11-16T22:33:41.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856, 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:41.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291564+0000", 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3995999999999998e-05 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291598+0000", 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5430000000000001e-06 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291603+0000", 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7660000000000004e-06 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291612+0000", 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3579999999999996e-06 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291616+0000", 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6769999999999999e-05 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291643+0000", 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559000000000002e-05 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291694+0000", 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9009000000000002e-05 2020-11-16T22:33:41.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291773+0000", 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071198067000000004 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362971+0000", 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00048011499999999998 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363451+0000", 2020-11-16T22:33:41.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1724000000000002e-05 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363493+0000", 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7949e-05 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363521+0000", 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071956856 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:19097 3.0 3:1e8917d7:::200.0000000e:head [write 2496718~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e128)", 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 187.566815558, 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998, 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.364 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344378+0000", 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3387999999999999e-05 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344412+0000", 2020-11-16T22:33:41.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2070000000000003e-06 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344417+0000", 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9299999999999992e-06 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344426+0000", 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0500000000000002e-06 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344430+0000", 2020-11-16T22:33:41.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0469e-05 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344490+0000", 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3390999999999998e-05 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344544+0000", 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0086000000000005e-05 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.344624+0000", 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055542677999999998 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.400166+0000", 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001392412 2020-11-16T22:33:41.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401559+0000", 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5942000000000002e-05 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401605+0000", 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9295e-05 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:33.401634+0000", 2020-11-16T22:33:41.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057255847999999998 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:41.374 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:41.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 158.36735822, 2020-11-16T22:33:41.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:41.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:41.381 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:41.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:41.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:41.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:41.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:41.392 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 129.88654756700001, 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:41.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:41.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:41.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:41.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:41.405 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.947115465000003, 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:41.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:41.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:41.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:41.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:41.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:53722 3.5 3:bec9163d:::200.00000015:head [write 2911097~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e181)", 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:41.417 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.724395047000002, 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061281549999999997, 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186799+0000", 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100505 2020-11-16T22:33:41.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186899+0000", 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5376e-05 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186915+0000", 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6409000000000001e-05 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186941+0000", 2020-11-16T22:33:41.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1097000000000001e-05 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.186952+0000", 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119407 2020-11-16T22:33:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.187072+0000", 2020-11-16T22:33:41.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.05766988 2020-11-16T22:33:41.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.244741+0000", 2020-11-16T22:33:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032219169999999999 2020-11-16T22:33:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.247963+0000", 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3133000000000001e-05 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.248017+0000", 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3825999999999995e-05 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:19.248080+0000", 2020-11-16T22:33:41.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061281549999999997 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54631 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.427 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.745760+0000", 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.165433945, 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054950938999999997, 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745760+0000", 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745760+0000", 2020-11-16T22:33:41.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3856e-05 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745784+0000", 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9489999999999998e-06 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745788+0000", 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8580000000000002e-06 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745795+0000", 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2789999999999999e-06 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745798+0000", 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4613000000000002e-05 2020-11-16T22:33:41.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745832+0000", 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3856999999999994e-05 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745916+0000", 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000172802 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746089+0000", 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073209779999999997 2020-11-16T22:33:41.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753410+0000", 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047136663000000002 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800547+0000", 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3533000000000002e-05 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800620+0000", 2020-11-16T22:33:41.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0550999999999994e-05 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800711+0000", 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054950938999999997 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54632 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.165367733, 2020-11-16T22:33:41.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054991695, 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:41.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:41.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5256999999999999e-05 2020-11-16T22:33:41.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745851+0000", 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7309999999999999e-06 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745855+0000", 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0190000000000001e-06 2020-11-16T22:33:41.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745862+0000", 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5679999999999998e-06 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745865+0000", 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6655000000000001e-05 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745901+0000", 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9556999999999997e-05 2020-11-16T22:33:41.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745991+0000", 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016009600000000001 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746151+0000", 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017664417000000002 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.763815+0000", 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.036864139999999997 2020-11-16T22:33:41.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800680+0000", 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7416999999999993e-05 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800767+0000", 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0837999999999997e-05 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800818+0000", 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054991695 2020-11-16T22:33:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54634 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.745992+0000", 2020-11-16T22:33:41.650 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.165202058999999, 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061758179000000003, 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745992+0000", 2020-11-16T22:33:41.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.745992+0000", 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5902999999999997e-05 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746038+0000", 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7950000000000002e-06 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746044+0000", 2020-11-16T22:33:41.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7544e-05 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746062+0000", 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8690000000000003e-06 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746067+0000", 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049210160000000003 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.750988+0000", 2020-11-16T22:33:41.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000102619 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.751091+0000", 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017340500000000001 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.751264+0000", 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012558332 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.763822+0000", 2020-11-16T22:33:41.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043800506000000003 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.807623+0000", 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6211000000000006e-05 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.807699+0000", 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0979000000000001e-05 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.807750+0000", 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061758179000000003 2020-11-16T22:33:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54635 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.746120+0000", 2020-11-16T22:33:41.660 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.165073954, 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.062617355, 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746120+0000", 2020-11-16T22:33:41.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746120+0000", 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7484000000000001e-05 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746167+0000", 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7660000000000001e-06 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746174+0000", 2020-11-16T22:33:41.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3363999999999999e-05 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746188+0000", 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.685e-06 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746192+0000", 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094693599999999996 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.755662+0000", 2020-11-16T22:33:41.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7406999999999998e-05 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.755759+0000", 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017742699999999999 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.755936+0000", 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0083946369999999999 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.764331+0000", 2020-11-16T22:33:41.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044273127000000002 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.808604+0000", 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5543000000000004e-05 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.808690+0000", 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7555000000000001e-05 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.808737+0000", 2020-11-16T22:33:41.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.062617355 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54636 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.746240+0000", 2020-11-16T22:33:41.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.164954282, 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054528707000000003, 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746240+0000", 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746240+0000", 2020-11-16T22:33:41.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7191000000000003e-05 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746287+0000", 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.605e-06 2020-11-16T22:33:41.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746294+0000", 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2678e-05 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746307+0000", 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4070000000000003e-06 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746311+0000", 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019820110000000001 2020-11-16T22:33:41.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.748293+0000", 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103861 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.748397+0000", 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000173392 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.748571+0000", 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095899650000000006 2020-11-16T22:33:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.758161+0000", 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042442779 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800603+0000", 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2440000000000005e-05 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800666+0000", 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000102378 2020-11-16T22:33:41.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800768+0000", 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054528707000000003 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54638 4.5 4:b0fbeeaa:::1000000188d.00000016:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.746365+0000", 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.164828436000001, 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064524524999999999, 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.681 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746365+0000", 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746365+0000", 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1378999999999998e-05 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746407+0000", 2020-11-16T22:33:41.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6370000000000002e-06 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746413+0000", 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1853e-05 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746425+0000", 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746430+0000", 2020-11-16T22:33:41.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014292437999999999 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.760722+0000", 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3320999999999998e-05 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.760795+0000", 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111105 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.760906+0000", 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058217210000000002 2020-11-16T22:33:41.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.766728+0000", 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044081046999999998 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.810809+0000", 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2539e-05 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.810862+0000", 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8232999999999999e-05 2020-11-16T22:33:41.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.810890+0000", 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064524524999999999 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54640 4.2 4:42f7da35:::1000000188d.00000018:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.746471+0000", 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.164723055, 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054158446999999998, 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746471+0000", 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746471+0000", 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3146000000000003e-05 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746514+0000", 2020-11-16T22:33:41.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1079999999999998e-06 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746520+0000", 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.172e-05 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746532+0000", 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8929999999999998e-06 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746536+0000", 2020-11-16T22:33:41.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2901000000000001e-05 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746579+0000", 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0113999999999998e-05 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746669+0000", 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000155847 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.746825+0000", 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032290589999999998 2020-11-16T22:33:41.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.750054+0000", 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.050386333999999998 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800440+0000", 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9363999999999997e-05 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800489+0000", 2020-11-16T22:33:41.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000139961 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.800629+0000", 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.054158446999999998 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74464 4.4 4:218b21fa:::1000000188d.00000001:head [write 3909465~41565 [1319@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:22.753216+0000", 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.157978031999999, 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.053142038000000003, 2020-11-16T22:33:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74464 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753216+0000", 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753216+0000", 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0496e-05 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753256+0000", 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7290000000000002e-06 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753260+0000", 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753267+0000", 2020-11-16T22:33:41.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.853e-06 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.753270+0000", 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00247482 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.755744+0000", 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012289199999999999 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.755867+0000", 2020-11-16T22:33:41.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019400600000000001 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.756061+0000", 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058890380000000001 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.761950+0000", 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044277481 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.806228+0000", 2020-11-16T22:33:41.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1572e-05 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.806279+0000", 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8523999999999996e-05 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:22.806358+0000", 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.053142038000000003 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.722201142999999, 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:41.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:41.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:33:41.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:33:41.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.720 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.72205276, 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:33:41.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:41.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:33:41.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:33:41.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:41.732 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:41.733 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:33:41.734 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:41.747+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:41.735 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:41.840 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:41.849+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:41.883 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:41.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:78786 4.0 e188/183 4:00fd5ff3:::1000000188d.0000000f:head v 188'10594, mlcod=188'10592)", 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.893637+0000", 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0049310049999999996, 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0049516439999999998, 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:41.884 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893637+0000", 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893637+0000", 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9568999999999998e-05 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893697+0000", 2020-11-16T22:33:41.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0809999999999996e-06 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893704+0000", 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.2960000000000004e-06 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893713+0000", 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0760000000000002e-06 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893718+0000", 2020-11-16T22:33:41.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9635999999999998e-05 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893758+0000", 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8025999999999999e-05 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:41.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.893786+0000", 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148684 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:41.890 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:33:41.891 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:33:41.891 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:41.893 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:41.935 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:41.950+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.036 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.050+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.095 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:42.095 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:42.095 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:42.095 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:42.095 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:42.105 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:42.136 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.151+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.231 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:42.237 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.252+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.258 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:42.258 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:42.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 336.95125490100003, 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:42.259 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:42.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:42.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 281.65866039700001, 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:42.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:42.277 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:42.278 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.361885323, 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:42.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:42.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:42.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:42.287 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.06350826400001, 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:42.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:42.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:42.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.298 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 162.18656025000001, 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:42.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:42.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:42.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 157.13396226399999, 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:42.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:42.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:42.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:42.314 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 157.13291893799999, 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:42.315 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:42.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:42.322 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.526122696000002, 2020-11-16T22:33:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:33:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.323 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:33:42.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:33:42.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:33:42.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.52603015, 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:33:42.333 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:33:42.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:33:42.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:33:42.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.525625049999999, 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.343 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.344 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:33:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.346 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:33:42.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.525541086, 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.351 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:33:42.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.523131207999999, 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.359 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:33:42.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.520404907, 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:42.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:33:42.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:33:42.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.374 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.518731084999999, 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:42.375 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:33:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:33:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:33:42.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.384 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.517990767000001, 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:33:42.385 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:33:42.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.082983611, 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:33:42.393 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:33:42.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:33:42.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:33:42.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.48425194999999999, 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.401 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:33:42.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:33:42.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.48417849099999999, 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.411 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:33:42.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:33:42.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:33:42.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.420 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.48357266599999998, 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:33:42.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:33:42.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:33:42.428 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.48341515699999998, 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:42.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:33:42.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:33:42.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:33:42.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.436 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.436 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.436 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:42.436 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:42.436 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:42.437 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:42.438 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:42.438 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:42.438 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:42.438 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:42.438 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:42.438 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:42.439 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:42.439 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:42.440 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.451+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.538 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.553+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.586 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:42.586 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:42.597 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:42.639 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:42.652+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:42.698 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2020-11-16T22:33:42.698 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2020-11-16T22:33:42.699 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.0 doing powercycle of ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:33:42.699 INFO:teuthology.orchestra.console:Power on smithi173 2020-11-16T22:33:42.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T22:33:42.722 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:33:42.740 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:42.754+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:42.780 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:42.780 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:42.792 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:42.841 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:42.855+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:42.935 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:42.935 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:42.942 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:42.957+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:42.945 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:33:43.042 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.057+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.102 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.103 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.114 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:43.144 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.158+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.245 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.259+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.262 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.263 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:43.345 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.359+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.410 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.411 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.421 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:43.445 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:43.459+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:43.546 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.560+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.565 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.565 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.576 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:43.647 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:43.661+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:43.748 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:43.762+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:43.759 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.759 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.769 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:43.849 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.864+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:43.909 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:43.910 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:43.920 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:43.960 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:43.975+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.052 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:44.066+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.067 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:44.067 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:44.079 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:44.153 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:44.168+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.229 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:44.229 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:44.240 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:44.283 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:44.297+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:44.354 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:44.369+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.389 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:44.390 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:44.401 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:44.440 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:44.455 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:44.469+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:44.553 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:44.554 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:44.556 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:44.569+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:44.563 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:44.591 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:44.612 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:44.612 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:44.613 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:44.613 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:44.613 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:44.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:44.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:44.656 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:44.671+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.711 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:44.711 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:33:44.711 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:33:44.711 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:33:44.712 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:44.723 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:44.757 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:44.770+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:44.858 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:44.872+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:44.926 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:44.926 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:33:44.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 226.93943599799999, 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:33:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:44.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:44.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:33:44.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:44.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:33:44.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:33:44.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:44.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.936 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 225.73128583100001, 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:33:44.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:33:44.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:33:44.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:33:44.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:44.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:44.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:33:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:33:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:33:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:33:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:33:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:33:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:33:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:33:45.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.355 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 194.65056997600001, 2020-11-16T22:33:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:33:45.356 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:45.357 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:33:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:33:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:33:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:33:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:33:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:45.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:33:45.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:33:45.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:45.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:33:45.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:33:45.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:33:45.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:45.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:33:45.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:33:45.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:45.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:33:45.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:45.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 194.65047453599999, 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:45.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:45.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:45.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:33:45.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:33:45.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:45.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:33:45.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:33:45.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:45.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:45.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:45.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:33:45.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:33:46.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:33:46.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 162.39566823600001, 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:46.194 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:46.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:46.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:46.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:46.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:46.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:33:46.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:46.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:33:46.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:33:46.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:46.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:33:46.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:33:46.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:46.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:46.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:46.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.615 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:33:46.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:46.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 133.91485758300001, 2020-11-16T22:33:46.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:33:46.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:46.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:46.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:46.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:46.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:46.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:33:46.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:33:46.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:47.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:33:47.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:47.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:33:47.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:47.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:33:47.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:33:47.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:47.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.236 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.975425481000002, 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:47.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:47.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:33:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:33:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:47.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:33:47.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:33:47.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:47.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:33:47.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:33:47.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:47.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:33:47.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:33:47.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:47.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:33:47.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:33:47.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:47.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:33:47.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:33:47.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:47.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:33:47.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:33:47.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:47.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:33:47.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:33:47.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:47.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:33:47.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:33:47.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:47.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:33:47.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:33:47.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:33:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:33:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:47.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:33:47.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:33:47.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:47.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:47.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:47.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.660 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:33:47.660 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:47.660 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.750511158999998, 2020-11-16T22:33:47.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:33:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:33:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:33:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:33:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:33:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:33:48.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:33:48.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:48.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:33:48.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:33:48.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:48.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:33:48.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:33:48.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:48.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:33:48.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:33:48.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:48.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:33:48.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:33:48.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:48.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:33:48.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:33:48.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:48.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.750362775999999, 2020-11-16T22:33:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:33:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:48.282 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:48.282 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:48.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:48.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:48.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:33:48.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:48.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:33:48.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:33:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:33:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:33:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:33:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:33:48.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59684 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1511949960000001, 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073231558000000002, 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:48.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3949000000000001e-05 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:48.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788363+0000", 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9190000000000002e-06 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788370+0000", 2020-11-16T22:33:48.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3614999999999999e-05 2020-11-16T22:33:48.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:48.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788383+0000", 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0329999999999999e-06 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788388+0000", 2020-11-16T22:33:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010747399999999999 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788496+0000", 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3537999999999999e-05 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788579+0000", 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013394500000000001 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788713+0000", 2020-11-16T22:33:48.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097288380000000001 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798442+0000", 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.063000930999999996 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:48.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:48.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:48.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861443+0000", 2020-11-16T22:33:48.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0006000000000002e-05 2020-11-16T22:33:48.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:49.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861493+0000", 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7309999999999999e-05 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861540+0000", 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073231558000000002 2020-11-16T22:33:49.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59686 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:33:49.116 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.150942498, 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073217435999999997, 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:49.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3176999999999997e-05 2020-11-16T22:33:49.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:49.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788605+0000", 2020-11-16T22:33:49.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3380000000000001e-06 2020-11-16T22:33:49.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788611+0000", 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5772000000000001e-05 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788627+0000", 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0280000000000002e-06 2020-11-16T22:33:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788631+0000", 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050616680000000001 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793692+0000", 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4448000000000002e-05 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793777+0000", 2020-11-16T22:33:49.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014774800000000001 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793925+0000", 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087258509999999997 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:49.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802650+0000", 2020-11-16T22:33:49.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058994670999999999 2020-11-16T22:33:49.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861645+0000", 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9487999999999996e-05 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861715+0000", 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4246999999999997e-05 2020-11-16T22:33:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861779+0000", 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073217435999999997 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59687 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1508289760000001, 2020-11-16T22:33:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.081678539999999994, 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:33:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.214e-05 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:49.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788717+0000", 2020-11-16T22:33:49.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-06 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788724+0000", 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1265e-05 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788735+0000", 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:49.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788739+0000", 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077344000000000002 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789512+0000", 2020-11-16T22:33:49.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1860999999999993e-05 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789604+0000", 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000151922 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789756+0000", 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017907330999999999 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807663+0000", 2020-11-16T22:33:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.062579546999999999 2020-11-16T22:33:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870243+0000", 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8853999999999997e-05 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870292+0000", 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1760000000000005e-05 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870353+0000", 2020-11-16T22:33:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.081678539999999994 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59689 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1506064550000001, 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072960093000000004, 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:33:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6764999999999997e-05 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788944+0000", 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9869999999999996e-06 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788950+0000", 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.201e-05 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788962+0000", 2020-11-16T22:33:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7219999999999999e-06 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788967+0000", 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072198080000000003 2020-11-16T22:33:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796187+0000", 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9411000000000003e-05 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796256+0000", 2020-11-16T22:33:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010552699999999999 2020-11-16T22:33:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796362+0000", 2020-11-16T22:33:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066535420000000001 2020-11-16T22:33:50.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.803015+0000", 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058680220999999998 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861695+0000", 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3822000000000001e-05 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861749+0000", 2020-11-16T22:33:50.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108278 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861858+0000", 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072960093000000004 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.161 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59690 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:33:50.162 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.150483828, 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.085068369000000005, 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:50.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:33:50.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:50.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4935000000000003e-05 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789065+0000", 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7840000000000004e-06 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789073+0000", 2020-11-16T22:33:50.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2632999999999999e-05 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789085+0000", 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0119999999999999e-06 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789089+0000", 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051023179999999998 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:50.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794192+0000", 2020-11-16T22:33:50.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6778000000000001e-05 2020-11-16T22:33:50.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:50.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794288+0000", 2020-11-16T22:33:50.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016072600000000001 2020-11-16T22:33:50.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794449+0000", 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013221134000000001 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807670+0000", 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.066341251000000004 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874012+0000", 2020-11-16T22:33:50.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2599999999999999e-05 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874054+0000", 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4198e-05 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874088+0000", 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.085068369000000005 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59692 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.150260125, 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.07251457, 2020-11-16T22:33:50.579 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:50.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:50.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:50.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:50.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1841000000000001e-05 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789286+0000", 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1589999999999998e-06 2020-11-16T22:33:50.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789292+0000", 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1071e-05 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789303+0000", 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789306+0000", 2020-11-16T22:33:50.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007556414 2020-11-16T22:33:50.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:50.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796863+0000", 2020-11-16T22:33:50.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105175 2020-11-16T22:33:50.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796968+0000", 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168818 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.797137+0000", 2020-11-16T22:33:50.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005866245 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.803003+0000", 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058566826000000002 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861570+0000", 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8752000000000003e-05 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861629+0000", 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012959199999999999 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861758+0000", 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.07251457 2020-11-16T22:33:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59693 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:33:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1501555809999999, 2020-11-16T22:33:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.076310798999999999, 2020-11-16T22:33:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2503999999999998e-05 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789391+0000", 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789397+0000", 2020-11-16T22:33:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3050999999999999e-05 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789410+0000", 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789415+0000", 2020-11-16T22:33:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090558879999999998 2020-11-16T22:33:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798471+0000", 2020-11-16T22:33:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7486999999999997e-05 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798568+0000", 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016673100000000001 2020-11-16T22:33:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798735+0000", 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056421550000000003 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.804377+0000", 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061202173999999998 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865579+0000", 2020-11-16T22:33:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0939999999999997e-05 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865630+0000", 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8824999999999999e-05 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865659+0000", 2020-11-16T22:33:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.076310798999999999 2020-11-16T22:33:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:51.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:51.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:51.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59694 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1500431739999999, 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084726294999999993, 2020-11-16T22:33:51.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:33:51.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6261999999999999e-05 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789507+0000", 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0690000000000004e-06 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:51.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789514+0000", 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2802e-05 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789527+0000", 2020-11-16T22:33:51.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4880000000000001e-06 2020-11-16T22:33:51.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:51.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789531+0000", 2020-11-16T22:33:51.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090725119999999996 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798604+0000", 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5232999999999998e-05 2020-11-16T22:33:51.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798659+0000", 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5279999999999996e-05 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798754+0000", 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011068545000000001 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809823+0000", 2020-11-16T22:33:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064295615 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874118+0000", 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1619999999999998e-05 2020-11-16T22:33:51.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874160+0000", 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6869e-05 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:51.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:52.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874187+0000", 2020-11-16T22:33:52.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084726294999999993 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59696 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:52.031 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1498025030000001, 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084587062000000005, 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:33:52.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6385999999999999e-05 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789748+0000", 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789754+0000", 2020-11-16T22:33:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2269000000000001e-05 2020-11-16T22:33:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789767+0000", 2020-11-16T22:33:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7829999999999998e-06 2020-11-16T22:33:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789771+0000", 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012456049 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802227+0000", 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3853e-05 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802291+0000", 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6032999999999993e-05 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802387+0000", 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074403860000000002 2020-11-16T22:33:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809828+0000", 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064393607000000005 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874221+0000", 2020-11-16T22:33:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0204999999999999e-05 2020-11-16T22:33:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:52.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874262+0000", 2020-11-16T22:33:52.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6851000000000001e-05 2020-11-16T22:33:52.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874288+0000", 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084587062000000005 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:52.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.448 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59697 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.149686231, 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084560730000000001, 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:52.449 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6283999999999999e-05 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:52.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789864+0000", 2020-11-16T22:33:52.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7440000000000001e-06 2020-11-16T22:33:52.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:52.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789871+0000", 2020-11-16T22:33:52.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1616e-05 2020-11-16T22:33:52.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789882+0000", 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0749999999999999e-06 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789886+0000", 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.015310302 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805197+0000", 2020-11-16T22:33:52.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1804999999999995e-05 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805258+0000", 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010677 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805365+0000", 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044668069999999997 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:52.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809832+0000", 2020-11-16T22:33:52.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064486257000000005 2020-11-16T22:33:52.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:52.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874318+0000", 2020-11-16T22:33:52.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6594999999999998e-05 2020-11-16T22:33:52.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874355+0000", 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3475000000000001e-05 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874378+0000", 2020-11-16T22:33:52.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084560730000000001 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.864 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59700 4.5 4:b0fbeeaa:::1000000188d.00000016:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.1494709849999998, 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.086085423999999994, 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:52.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:52.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:33:52.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4501e-05 2020-11-16T22:33:52.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:52.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:52.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:52.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790077+0000", 2020-11-16T22:33:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.049e-06 2020-11-16T22:33:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790083+0000", 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1250000000000001e-05 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790095+0000", 2020-11-16T22:33:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9380000000000002e-06 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790099+0000", 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017588217 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807687+0000", 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2090999999999999e-05 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807749+0000", 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100796 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807850+0000", 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034151440000000002 2020-11-16T22:33:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:33:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.811265+0000", 2020-11-16T22:33:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064766985999999999 2020-11-16T22:33:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876032+0000", 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7299000000000002e-05 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876079+0000", 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9153000000000003e-05 2020-11-16T22:33:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876118+0000", 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.086085423999999994 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:53.281 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:53.282 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:33:53.282 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:53.283 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T22:33:53.293 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:53.297+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:53.294 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:53.383 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:53.397+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:53.443 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.443 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:62978 3.7 3:f376ae95:::200.00000016:head [write 3529951~1927 [fadvise_dontneed] in=1927b] snapc 0=[] ondisk+write+known_if_redirected+full_force e189)", 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:53.454656+0000", 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.003554916, 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.003573381, 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:53.444 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454656+0000", 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454656+0000", 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0964999999999996e-05 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454727+0000", 2020-11-16T22:33:53.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8539999999999992e-06 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454737+0000", 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6939000000000002e-05 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454754+0000", 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.435e-06 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:53.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454762+0000", 2020-11-16T22:33:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2485999999999999e-05 2020-11-16T22:33:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454814+0000", 2020-11-16T22:33:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00013404700000000001 2020-11-16T22:33:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:53.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:53.454948+0000", 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00029172600000000002 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:53.651 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:33:53.652 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:33:53.652 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:53.652 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2020-11-16T22:33:53.654 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:53.654 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:53.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:53.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:53.656 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:53.656 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:53.667+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:53.753 DEBUG:teuthology.orchestra.console:Waiting for login prompt on smithi173 2020-11-16T22:33:53.754 DEBUG:teuthology.orchestra.console:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi173 2020-11-16T22:33:53.763 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:53.768+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:53.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~147s 2020-11-16T22:33:53.813 DEBUG:teuthology.orchestra.console:expect: smithi173 login 2020-11-16T22:33:53.841 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:53.842 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:53.852 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:53.855 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:53.870+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:53.956 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:53.970+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.014 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:33:54.036 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.036 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.046 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:54.086 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:54.101+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.157 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:54.172+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.193 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.194 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.203 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:54.258 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:54.273+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.350 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.350 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.359 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:54.374+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.360 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:54.460 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:54.475+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:54.517 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.518 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.528 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:54.571 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:54.586+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:54.662 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:54.676+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:54.665 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.665 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.679 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:54.762 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:54.777+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:54.863 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:54.877+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:54.865 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:54.866 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:54.877 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:54.964 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:54.979+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.021 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:55.021 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:55.031 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:55.065 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:55.080+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:55.166 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:55.180+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.170 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:55.170 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:55.181 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:55.267 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:55.281+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.367 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:55.382+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.379 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:55.379 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:55.389 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:55.474 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:55.488+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.570 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:55.584+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:55.575 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:55.575 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:55.586 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:33:55.656 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:33:55.671 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:55.685+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:55.734 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:55.734 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:55.746 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:33:55.790 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:55.805+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:55.807 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:55.827 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:55.828 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:55.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:55.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:33:55.828 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:33:55.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:55.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:55.872 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:55.887+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:55.894 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:33:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:33:55.894 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:33:55.895 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:55.905 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:33:55.988 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:56.003+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:56.026 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:33:56.074 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:56.089+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:56.098 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:33:56.098 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:33:56.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:33:56.098 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:33:56.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:24 4.3 4:d2dbd4ac:::10000000217.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e85)", 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 350.79084266500001, 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999, 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:56.099 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 24 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.100 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.320576+0000", 2020-11-16T22:33:56.101 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.011013545 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331590+0000", 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0781000000000001e-05 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.102 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331601+0000", 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.8158000000000001e-05 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:56.103 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331619+0000", 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0115000000000001e-05 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331629+0000", 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7280999999999993e-05 2020-11-16T22:33:56.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331716+0000", 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00021701700000000001 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:56.105 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.331933+0000", 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.031401150000000003 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363335+0000", 2020-11-16T22:33:56.106 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4797999999999999e-05 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363389+0000", 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3770999999999996e-05 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.107 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:28:05.363453+0000", 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.042876615999999999 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.108 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 295.49824816099999, 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:56.109 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.110 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:33:56.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.112 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:33:56.113 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.114 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:33:56.115 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:33:56.116 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:33:56.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.116 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:56.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:33:56.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:56.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:56.321 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 237.20147308700001, 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:56.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:56.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:56.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:33:56.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:56.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:33:56.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:33:56.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:33:56.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:33:56.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:33:56.737 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 236.90309602799999, 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.738 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:56.739 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:56.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:56.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.739 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:56.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:33:56.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:33:56.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:56.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:33:56.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:56.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:56.947 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:56.947 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:33:57.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:33:57.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:57.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:33:57.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:33:57.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:33:57.155 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.155 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:57.155 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.358 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:33:57.358 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:57.358 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 176.026148014, 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:57.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:33:57.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:33:57.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:57.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:33:57.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:33:57.566 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.566 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.566 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:57.566 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:33:57.566 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.567 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:57.568 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.569 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 170.97355002800001, 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:57.570 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:57.571 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:33:57.572 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.573 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:33:57.574 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.575 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.576 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:57.576 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:33:57.576 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:33:57.576 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:33:57.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:33:57.780 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 170.972506702, 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:57.781 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:57.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:33:57.986 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.987 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:57.988 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:33:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:57.990 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:33:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:33:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:57.991 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:58.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.194 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.365710460000003, 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:58.195 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:33:58.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:58.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:33:58.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:33:58.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:58.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:33:58.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:58.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:33:58.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:33:58.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:58.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:33:58.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:33:58.610 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:58.610 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.365617913999998, 2020-11-16T22:33:58.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:58.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:58.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:33:58.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:33:58.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:58.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:33:58.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:33:58.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:33:58.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:58.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:33:58.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:33:58.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:58.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:58.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:58.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:33:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:33:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.365212814000003, 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:33:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:33:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:33:59.234 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:33:59.234 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.234 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.234 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:33:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.236 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.237 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:33:59.238 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.239 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.239 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.239 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:33:59.442 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:59.442 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.365128849999998, 2020-11-16T22:33:59.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:59.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:33:59.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:33:59.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.362718972000003, 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.451 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:33:59.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:33:59.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:33:59.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:33:59.459 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.359992671000001, 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:33:59.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:33:59.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.464 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:33:59.465 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.466 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.358318849, 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.468 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.469 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:33:59.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:33:59.472 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.473 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.474 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:33:59.475 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.476 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:33:59.477 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.478 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.357578531000001, 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:33:59.479 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.480 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.481 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:33:59.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:33:59.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.487 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.922571374999997, 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.488 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:33:59.489 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:33:59.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:33:59.493 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:33:59.494 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.323839714, 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.496 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.497 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.498 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:33:59.499 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.500 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:33:59.501 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.502 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:33:59.503 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.504 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.505 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:33:59.506 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.323766255000001, 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.507 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:33:59.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:33:59.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:33:59.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.516 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.32316043, 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:33:59.517 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.518 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.519 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:33:59.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.521 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:33:59.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.523 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:33:59.524 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:59.525 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 14.323002921, 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:33:59.526 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:33:59.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.528 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:33:59.529 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.530 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:33:59.531 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:33:59.532 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.533 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:33:59.534 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:33:59.534 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:33:59.534 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:33:59.536 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:33:59.537 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:33:59.537 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:59.552+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:59.636 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:59.651+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:33:59.684 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:33:59.684 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:33:59.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.684 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:83174 4.7 4:eeb50346:::1000000188d.00000000:head [read 2121728~131072 [1988@4194304] out=131072b] snapc 0=[] ondisk+read+known_if_redirected e191)", 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:59.698812+0000", 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000368334, 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037799699999999998, 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:33:59.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 83174 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698812+0000", 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698812+0000", 2020-11-16T22:33:59.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1387999999999999e-05 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698834+0000", 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9649999999999999e-06 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698839+0000", 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1984000000000001e-05 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698851+0000", 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5910000000000003e-06 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698856+0000", 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7191999999999999e-05 2020-11-16T22:33:59.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698893+0000", 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0420000000000005e-05 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:33:59.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698984+0000", 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020645700000000001 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.699190+0000", 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037799699999999998 2020-11-16T22:33:59.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:83175 4.7 4:eeb50346:::1000000188d.00000000:head [write 2056192~36864 [1988@4194304] in=36864b] snapc 1=[] ondisk+write+known_if_redirected e191)", 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:59.698886+0000", 2020-11-16T22:33:59.693 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00029414799999999999, 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036526899999999999, 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:33:59.694 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 83175 2020-11-16T22:33:59.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.695 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:33:59.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:33:59.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698886+0000", 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698886+0000", 2020-11-16T22:33:59.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9505999999999997e-05 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698926+0000", 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.766e-06 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698930+0000", 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0670000000000003e-06 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:33:59.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698937+0000", 2020-11-16T22:33:59.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9340000000000002e-06 2020-11-16T22:33:59.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:33:59.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.698940+0000", 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027743400000000001 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:59.699217+0000", 2020-11-16T22:33:59.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033070700000000002 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2020-11-16T22:33:59.701 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:33:59.702 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:33:59.727 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:33:59.745 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:59.760+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:59.749 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:33:59.749 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:33:59.750 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:33:59.750 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:33:59.750 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:33:59.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:33:59.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:33:59.838 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:33:59.852+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:33:59.847 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:33:59.848 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:33:59.857 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:33:59.938 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:33:59.953+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:00.040 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:00.054+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:00.053 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.053 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.063 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:00.141 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.155+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.202 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.202 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.213 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:00.241 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.256+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.343 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.357+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~162s 2020-11-16T22:34:00.361 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.362 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.374 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:00.444 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.459+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.533 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.533 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.544 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:00.559+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:00.546 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:00.645 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.660+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.691 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.691 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.703 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:00.787 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.803+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.847 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:00.863+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:00.892 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:00.892 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:00.902 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:00.948 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:00.962+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.048 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.063+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.122 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:01.122 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:01.134 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:01.149 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:01.165+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:01.250 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.264+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.285 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:01.285 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:01.296 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:01.350 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:01.365+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:01.450 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:01.466+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:01.481 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:01.481 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:01.493 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:01.537 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:34:01.560 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:01.575+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:01.643 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:01.643 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:01.653 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:01.660 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.675+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.753 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:01.761 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.776+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:01.821 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:01.862 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.877+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.903 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:01.925 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:01.925 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:01.925 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:01.925 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:01.927 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:01.927 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:01.963 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:01.978+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:01.969 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:01.970 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:01.970 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:01.970 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:01.970 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:01.980 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:02.039 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:34:02.064 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:02.079+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:02.134 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:02.134 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:02.135 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 244.14755137399999, 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:02.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:02.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:02.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:02.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:02.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 242.939401207, 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:02.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:02.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:02.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:02.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:02.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:02.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:02.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:02.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:02.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.361 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30226 4.1 4:8c3af0c3:::1000000188a.00000000:head [write 1191936~32768 [251@4194304]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 211.85868535200001, 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004, 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:02.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30226 2020-11-16T22:34:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:02.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:34:02.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288934+0000", 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0151999999999998e-05 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288974+0000", 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3939999999999998e-06 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:02.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288978+0000", 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1790000000000002e-06 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288986+0000", 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2370000000000003e-06 2020-11-16T22:34:02.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.288990+0000", 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7187000000000002e-05 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289027+0000", 2020-11-16T22:34:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5014999999999997e-05 2020-11-16T22:34:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289122+0000", 2020-11-16T22:34:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000132442 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289254+0000", 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015235139999999999 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.290778+0000", 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072004030999999996 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362782+0000", 2020-11-16T22:34:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6789000000000001e-05 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362839+0000", 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9478000000000002e-05 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362898+0000", 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073964418000000004 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:30227 4.5 4:a611fa68:::1000000188a.00000002:head [write 149935~43811 [251@388401],write 1652358~38724 [251@388401]] snapc 1=[] ondisk+write+known_if_redirected e128)", 2020-11-16T22:34:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:34:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 211.85858991200001, 2020-11-16T22:34:02.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994, 2020-11-16T22:34:02.982 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:02.982 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:02.982 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30227 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:02.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289029+0000", 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116928 2020-11-16T22:34:02.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289146+0000", 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3889999999999999e-06 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289154+0000", 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2048999999999999e-05 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289166+0000", 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:34:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289171+0000", 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1831000000000001e-05 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289203+0000", 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3662000000000002e-05 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289257+0000", 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118662 2020-11-16T22:34:02.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.289375+0000", 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019613949999999999 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:02.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.291337+0000", 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.071540553000000007 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362877+0000", 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1650999999999996e-05 2020-11-16T22:34:02.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.362949+0000", 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:02.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:30.363002+0000", 2020-11-16T22:34:03.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073972610999999994 2020-11-16T22:34:03.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 179.603783612, 2020-11-16T22:34:03.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:03.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:03.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:03.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:03.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:03.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:03.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:03.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:03.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:03.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:03.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 151.12297295900001, 2020-11-16T22:34:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:03.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:03.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:03.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:03.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:04.030 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:04.030 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.183540856999997, 2020-11-16T22:34:04.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:04.030 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:04.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:04.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:04.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:04.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:04.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:04.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:04.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.447 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.958626535000001, 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:04.448 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:04.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:04.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:04.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.659 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.958478151999998, 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:04.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:04.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:04.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:04.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:04.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:04.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:04.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59684 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.359310371999999, 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073231558000000002, 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:34:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788309+0000", 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3949000000000001e-05 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788363+0000", 2020-11-16T22:34:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9190000000000002e-06 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788370+0000", 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3614999999999999e-05 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788383+0000", 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0329999999999999e-06 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788388+0000", 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010747399999999999 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788496+0000", 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3537999999999999e-05 2020-11-16T22:34:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788579+0000", 2020-11-16T22:34:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013394500000000001 2020-11-16T22:34:05.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:05.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788713+0000", 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097288380000000001 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798442+0000", 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.063000930999999996 2020-11-16T22:34:05.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861443+0000", 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0006000000000002e-05 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861493+0000", 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7309999999999999e-05 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861540+0000", 2020-11-16T22:34:05.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073231558000000002 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59686 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:05.494 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:34:05.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.359057874000001, 2020-11-16T22:34:05.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073217435999999997, 2020-11-16T22:34:05.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788561+0000", 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3176999999999997e-05 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788605+0000", 2020-11-16T22:34:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3380000000000001e-06 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788611+0000", 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5772000000000001e-05 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788627+0000", 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0280000000000002e-06 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788631+0000", 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050616680000000001 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793692+0000", 2020-11-16T22:34:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4448000000000002e-05 2020-11-16T22:34:05.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:05.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793777+0000", 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014774800000000001 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.793925+0000", 2020-11-16T22:34:05.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087258509999999997 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802650+0000", 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058994670999999999 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861645+0000", 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9487999999999996e-05 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861715+0000", 2020-11-16T22:34:05.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4246999999999997e-05 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861779+0000", 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.073217435999999997 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.910 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:05.911 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:05.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:05.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:05.911 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59687 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:34:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358944352000002, 2020-11-16T22:34:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.081678539999999994, 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788675+0000", 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.214e-05 2020-11-16T22:34:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788717+0000", 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-06 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788724+0000", 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1265e-05 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788735+0000", 2020-11-16T22:34:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788739+0000", 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077344000000000002 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789512+0000", 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1860999999999993e-05 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789604+0000", 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000151922 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789756+0000", 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017907330999999999 2020-11-16T22:34:06.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807663+0000", 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.062579546999999999 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:06.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870243+0000", 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8853999999999997e-05 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870292+0000", 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1760000000000005e-05 2020-11-16T22:34:06.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.870353+0000", 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.081678539999999994 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:06.124 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59689 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358721831, 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072960093000000004, 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788897+0000", 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6764999999999997e-05 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788944+0000", 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9869999999999996e-06 2020-11-16T22:34:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788950+0000", 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.201e-05 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788962+0000", 2020-11-16T22:34:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7219999999999999e-06 2020-11-16T22:34:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.788967+0000", 2020-11-16T22:34:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072198080000000003 2020-11-16T22:34:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796187+0000", 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9411000000000003e-05 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796256+0000", 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010552699999999999 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796362+0000", 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066535420000000001 2020-11-16T22:34:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.803015+0000", 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058680220999999998 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861695+0000", 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3822000000000001e-05 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861749+0000", 2020-11-16T22:34:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108278 2020-11-16T22:34:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861858+0000", 2020-11-16T22:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.072960093000000004 2020-11-16T22:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59690 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358599204000001, 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.085068369000000005, 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789020+0000", 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4935000000000003e-05 2020-11-16T22:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789065+0000", 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7840000000000004e-06 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789073+0000", 2020-11-16T22:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2632999999999999e-05 2020-11-16T22:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:06.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789085+0000", 2020-11-16T22:34:06.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0119999999999999e-06 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789089+0000", 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051023179999999998 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794192+0000", 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6778000000000001e-05 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794288+0000", 2020-11-16T22:34:06.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016072600000000001 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.794449+0000", 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013221134000000001 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807670+0000", 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.066341251000000004 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874012+0000", 2020-11-16T22:34:06.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2599999999999999e-05 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:06.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874054+0000", 2020-11-16T22:34:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4198e-05 2020-11-16T22:34:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874088+0000", 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.085068369000000005 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59692 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358375501000001, 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.07251457, 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789244+0000", 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1841000000000001e-05 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789286+0000", 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1589999999999998e-06 2020-11-16T22:34:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789292+0000", 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1071e-05 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789303+0000", 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789306+0000", 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007556414 2020-11-16T22:34:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796863+0000", 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105175 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.796968+0000", 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000168818 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.797137+0000", 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005866245 2020-11-16T22:34:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.803003+0000", 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058566826000000002 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861570+0000", 2020-11-16T22:34:07.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8752000000000003e-05 2020-11-16T22:34:07.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861629+0000", 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012959199999999999 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.861758+0000", 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.07251457 2020-11-16T22:34:07.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59693 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:34:07.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358270956999998, 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.076310798999999999, 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:07.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789348+0000", 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2503999999999998e-05 2020-11-16T22:34:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:07.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789391+0000", 2020-11-16T22:34:07.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:34:07.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789397+0000", 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3050999999999999e-05 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:07.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789410+0000", 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789415+0000", 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090558879999999998 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798471+0000", 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7486999999999997e-05 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:07.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798568+0000", 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016673100000000001 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798735+0000", 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056421550000000003 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.804377+0000", 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.061202173999999998 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:07.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865579+0000", 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0939999999999997e-05 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865630+0000", 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8824999999999999e-05 2020-11-16T22:34:07.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.865659+0000", 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.076310798999999999 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:07.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59694 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.358158549999999, 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084726294999999993, 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:07.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:07.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789461+0000", 2020-11-16T22:34:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6261999999999999e-05 2020-11-16T22:34:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789507+0000", 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0690000000000004e-06 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789514+0000", 2020-11-16T22:34:07.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2802e-05 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789527+0000", 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4880000000000001e-06 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789531+0000", 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090725119999999996 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798604+0000", 2020-11-16T22:34:07.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5232999999999998e-05 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798659+0000", 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5279999999999996e-05 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.798754+0000", 2020-11-16T22:34:07.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011068545000000001 2020-11-16T22:34:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:07.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809823+0000", 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064295615 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874118+0000", 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1619999999999998e-05 2020-11-16T22:34:07.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874160+0000", 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6869e-05 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:08.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874187+0000", 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084726294999999993 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.001 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59696 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.357917878999999, 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084587062000000005, 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789701+0000", 2020-11-16T22:34:08.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6385999999999999e-05 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789748+0000", 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.64e-06 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789754+0000", 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2269000000000001e-05 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789767+0000", 2020-11-16T22:34:08.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7829999999999998e-06 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789771+0000", 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012456049 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802227+0000", 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3853e-05 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802291+0000", 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6032999999999993e-05 2020-11-16T22:34:08.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.802387+0000", 2020-11-16T22:34:08.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074403860000000002 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809828+0000", 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064393607000000005 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874221+0000", 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0204999999999999e-05 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874262+0000", 2020-11-16T22:34:08.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6851000000000001e-05 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874288+0000", 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084587062000000005 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:08.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59697 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.357801606999999, 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084560730000000001, 2020-11-16T22:34:08.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:08.215 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:08.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789818+0000", 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6283999999999999e-05 2020-11-16T22:34:08.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789864+0000", 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7440000000000001e-06 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:08.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789871+0000", 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1616e-05 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789882+0000", 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0749999999999999e-06 2020-11-16T22:34:08.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.789886+0000", 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.015310302 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805197+0000", 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1804999999999995e-05 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805258+0000", 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010677 2020-11-16T22:34:08.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.805365+0000", 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044668069999999997 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.809832+0000", 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064486257000000005 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874318+0000", 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6594999999999998e-05 2020-11-16T22:34:08.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874355+0000", 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3475000000000001e-05 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.874378+0000", 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.084560730000000001 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:59700 4.5 4:b0fbeeaa:::1000000188d.00000016:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.357586360999999, 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.086085423999999994, 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:08.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790033+0000", 2020-11-16T22:34:08.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4501e-05 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790077+0000", 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.049e-06 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790083+0000", 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1250000000000001e-05 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790095+0000", 2020-11-16T22:34:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9380000000000002e-06 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.790099+0000", 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017588217 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807687+0000", 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2090999999999999e-05 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807749+0000", 2020-11-16T22:34:08.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100796 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.807850+0000", 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034151440000000002 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.811265+0000", 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.064766985999999999 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876032+0000", 2020-11-16T22:34:08.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7299000000000002e-05 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876079+0000", 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9153000000000003e-05 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:08.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:41.876118+0000", 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.086085423999999994 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:08.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:08.438 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:08.438 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:08.439 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:34:08.440 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:08.453+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:08.441 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:08.539 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:08.554+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:08.587 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:08.587 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:08.588 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:34:08.588 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:08.589 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:08.597 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:08.611 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:08.612 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:08.612 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:08.612 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:08.612 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:08.614 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:08.614 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:08.640 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:08.654+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:08.741 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:08.741 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:08.742 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:08.755+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:08.751 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:08.841 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:08.855+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:08.913 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:08.914 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:08.923 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:08.942 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:08.956+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.043 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:09.058+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.060 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:34:09.070 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.071 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.082 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:09.144 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:09.158+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:09.221 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.221 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.231 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:09.245 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:09.260+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~171s 2020-11-16T22:34:09.346 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:09.360+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:09.417 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.417 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.428 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:09.471 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:09.486+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.547 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:09.561+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:09.576 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.577 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.586 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:09.647 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:09.662+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:09.748 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:09.763+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.773 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.773 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.783 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:09.866 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:09.881+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:09.949 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:09.963+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:09.975 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:09.975 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:09.987 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:10.050 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:10.065+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:10.128 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:10.128 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:10.139 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:10.183 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.198+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.253 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:10.267+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:10.279 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:10.280 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:10.290 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:10.353 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:10.368+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:10.454 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.469+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.466 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:10.466 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:10.476 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:10.555 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:10.570+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:10.615 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:10.627 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:10.627 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:10.638 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:10.656 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.671+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.757 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.772+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.764 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:10.787 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:10.789 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:10.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:10.828 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:10.829 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:10.829 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:10.829 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:10.829 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:10.839 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:10.858 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.872+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.958 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:10.973+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:10.997 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:10.997 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:10.997 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:10.997 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 310.39741480999999, 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:10.998 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:10.999 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:11.000 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.001 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:11.002 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.003 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:11.004 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.005 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:11.006 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.007 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 252.10063973600001, 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:11.008 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.009 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.010 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.011 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:11.012 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.013 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.014 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:11.015 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:11.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.015 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.015 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:11.015 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:11.219 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:11.220 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 251.80226267699999, 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.221 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:11.222 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:11.223 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.223 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.223 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.223 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:11.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:11.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:11.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:11.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:11.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:11.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:11.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.634 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.638 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 190.92531466299999, 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.639 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.640 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.641 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.642 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:11.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.644 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.872716677, 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:34:11.647 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.648 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.649 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:34:11.650 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.651 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:34:11.652 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.653 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:34:11.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:34:11.655 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 185.871673351, 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:11.656 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:34:11.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:34:11.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.264877108999997, 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.664 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.665 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:11.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:11.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:11.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.672 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:11.673 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:11.674 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.264784562999999, 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:11.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:11.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:11.682 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.683 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.684 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.264379462999997, 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.685 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.686 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:11.687 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.688 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:11.689 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.690 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.691 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.264295498999999, 2020-11-16T22:34:11.693 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:11.694 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.695 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:11.696 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.697 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.701 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.261885620999998, 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:11.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:11.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:11.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:11.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.259159320000002, 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:11.710 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.711 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:11.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:11.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:11.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:11.718 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.257485498000001, 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:11.719 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:11.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:11.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:11.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:11.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.728 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.729 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.729 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:11.729 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:11.729 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.729 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 48.256745180000003, 2020-11-16T22:34:11.730 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:11.731 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.732 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:34:11.733 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:34:11.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.735 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:34:11.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.737 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.738 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 47.821738023999998, 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.739 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.740 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:11.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:11.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.744 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:11.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.746 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.223006363, 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:11.747 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:11.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.751 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:11.752 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.753 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.754 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:11.755 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.756 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.757 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.222932904, 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.758 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:11.760 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.761 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:11.762 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.763 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:11.764 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.765 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:11.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.767 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.222327078999999, 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:11.768 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.772 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.773 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:11.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.775 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.776 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 29.222169569999998, 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.777 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.778 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:11.779 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.780 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:11.781 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.782 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.783 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.784 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 5.0486191500000004, 2020-11-16T22:34:11.785 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:11.786 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.787 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:11.788 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.789 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:11.790 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:11.791 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.792 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:11.793 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:11.794 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:11.795 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:11.796 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:11.796 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:34:11.798 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:11.798 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:11.811+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:11.897 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:11.912+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:11.960 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:11.960 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:85814 4.5 4:ab64ebfc:::1000000188d.00000009:head [zero 3192609~5607] snapc 1=[] ondisk+write+known_if_redirected e193)", 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:11.951970+0000", 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.023571584999999999, 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023588438999999999, 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:11.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 85814 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:11.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.951970+0000", 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:11.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.951970+0000", 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2517e-05 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.952013+0000", 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0710000000000001e-06 2020-11-16T22:34:11.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.952022+0000", 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7297000000000001e-05 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.952039+0000", 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5950000000000001e-06 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.952047+0000", 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01388436 2020-11-16T22:34:11.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.965931+0000", 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011442099999999999 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:11.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:11.966046+0000", 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014075261 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:11.968 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:34:11.969 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:34:11.969 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:11.970 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:12.015 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.030+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:12.100 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.114+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:12.120 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:12.120 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:12.120 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:12.120 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:12.120 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:12.132 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:12.200 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:12.215+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:12.301 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.315+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:12.331 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:12.331 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 254.34473420200001, 2020-11-16T22:34:12.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:12.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:12.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:12.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:12.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:12.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:12.342 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 253.136584035, 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:12.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:12.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:12.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:12.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:12.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 189.80096644, 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:12.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:12.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:12.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:12.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 161.320155787, 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:12.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:12.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:12.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.380723685000007, 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:12.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:12.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:12.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:12.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.155809363000003, 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:12.393 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:12.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:12.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:12.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:12.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.15566098, 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.403 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:12.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:12.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:12.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:12.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:12.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66577 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 2104@3880336] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.414 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.383379294, 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634, 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6947000000000001e-05 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961450+0000", 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7630000000000003e-06 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961456+0000", 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2191999999999999e-05 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961469+0000", 2020-11-16T22:34:12.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961474+0000", 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5272999999999999e-05 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961510+0000", 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100571 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961610+0000", 2020-11-16T22:34:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176787 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961787+0000", 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085900679999999993 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970377+0000", 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.091642535999999997 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062020+0000", 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0728000000000004e-05 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062100+0000", 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2172000000000001e-05 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062152+0000", 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66578 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3832996209999999, 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003, 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3958e-05 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961526+0000", 2020-11-16T22:34:12.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.014e-06 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961530+0000", 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7959999999999998e-06 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961537+0000", 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.897e-06 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961540+0000", 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5605000000000002e-05 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961576+0000", 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5517000000000003e-05 2020-11-16T22:34:12.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961641+0000", 2020-11-16T22:34:12.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010893899999999999 2020-11-16T22:34:12.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961750+0000", 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073697459999999999 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969120+0000", 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088765291999999996 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057885+0000", 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6371000000000004e-05 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057972+0000", 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013677599999999999 2020-11-16T22:34:12.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058108+0000", 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.435 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66579 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.383233207, 2020-11-16T22:34:12.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999, 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:12.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6403000000000001e-05 2020-11-16T22:34:12.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961595+0000", 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9859999999999996e-06 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961599+0000", 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961606+0000", 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4480000000000001e-06 2020-11-16T22:34:12.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961609+0000", 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049254529999999998 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966534+0000", 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7503999999999997e-05 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966592+0000", 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7417999999999995e-05 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966689+0000", 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095498420000000001 2020-11-16T22:34:12.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062187+0000", 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4015000000000003e-05 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062251+0000", 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1647e-05 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062323+0000", 2020-11-16T22:34:12.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2119e-05 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062375+0000", 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.446 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66580 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3831656329999999, 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688, 2020-11-16T22:34:12.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4732999999999999e-05 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961661+0000", 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961665+0000", 2020-11-16T22:34:12.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9530000000000002e-06 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961672+0000", 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961682+0000", 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0902999999999998e-05 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961723+0000", 2020-11-16T22:34:12.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8277999999999999e-05 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961791+0000", 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010717000000000001 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961898+0000", 2020-11-16T22:34:12.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10533300399999999 2020-11-16T22:34:12.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067231+0000", 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1401e-05 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067243+0000", 2020-11-16T22:34:12.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1019999999999993e-05 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067324+0000", 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5339e-05 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067379+0000", 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66581 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.383082967, 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006, 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.458 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7623999999999998e-05 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961747+0000", 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2020-11-16T22:34:12.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961751+0000", 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0149999999999998e-06 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961758+0000", 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2649999999999998e-06 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961762+0000", 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4431000000000002e-05 2020-11-16T22:34:12.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961807+0000", 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035119090000000001 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965319+0000", 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012228 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965441+0000", 2020-11-16T22:34:12.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092836962999999995 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058278+0000", 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013207889999999999 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059599+0000", 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2479999999999996e-05 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059681+0000", 2020-11-16T22:34:12.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5322999999999995e-05 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059776+0000", 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.467 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66582 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3829871840000001, 2020-11-16T22:34:12.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302, 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0484e-05 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961865+0000", 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1609999999999997e-06 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961873+0000", 2020-11-16T22:34:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2751999999999999e-05 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961885+0000", 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4719999999999997e-06 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961890+0000", 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080703010000000002 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969960+0000", 2020-11-16T22:34:12.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7603999999999999e-05 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970018+0000", 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010071199999999999 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970118+0000", 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088442494999999996 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058561+0000", 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037181409999999999 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062279+0000", 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3547000000000001e-05 2020-11-16T22:34:12.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062333+0000", 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3633000000000004e-05 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062416+0000", 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.478 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66583 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.382864616, 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072, 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.479 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:12.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6323999999999999e-05 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961984+0000", 2020-11-16T22:34:12.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.109e-06 2020-11-16T22:34:12.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961991+0000", 2020-11-16T22:34:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2452999999999999e-05 2020-11-16T22:34:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962003+0000", 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962008+0000", 2020-11-16T22:34:12.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046175440000000003 2020-11-16T22:34:12.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966625+0000", 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0504999999999998e-05 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966686+0000", 2020-11-16T22:34:12.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3917000000000002e-05 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966780+0000", 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100608974 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067389+0000", 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2806999999999999e-05 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067431+0000", 2020-11-16T22:34:12.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6773999999999994e-05 2020-11-16T22:34:12.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067508+0000", 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1255000000000003e-05 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067560+0000", 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66585 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3826340569999997, 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994, 2020-11-16T22:34:12.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5401000000000001e-05 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962213+0000", 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1690000000000003e-06 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962221+0000", 2020-11-16T22:34:12.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2514000000000001e-05 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962233+0000", 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.378e-06 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962238+0000", 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032176309999999999 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965456+0000", 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101195 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965557+0000", 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149464 2020-11-16T22:34:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965707+0000", 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092509202999999998 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058216+0000", 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013792990000000001 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059595+0000", 2020-11-16T22:34:12.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4564e-05 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059670+0000", 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013108299999999999 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059801+0000", 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66586 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3825145030000003, 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146, 2020-11-16T22:34:12.503 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5766999999999997e-05 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962333+0000", 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4960000000000001e-06 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962340+0000", 2020-11-16T22:34:12.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.313e-05 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962353+0000", 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.763e-06 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962358+0000", 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080385160000000008 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970396+0000", 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.08724875 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057645+0000", 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018116900000000001 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057826+0000", 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063036259999999997 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064130+0000", 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030772299999999998 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064438+0000", 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5346000000000003e-05 2020-11-16T22:34:12.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064483+0000", 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8860000000000002e-05 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064512+0000", 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66587 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3823994910000001, 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938, 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.514 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6044000000000002e-05 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962449+0000", 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9759999999999998e-06 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962456+0000", 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2969999999999999e-05 2020-11-16T22:34:12.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962469+0000", 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.977e-06 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962474+0000", 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079504459999999999 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970424+0000", 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.087276322000000003 2020-11-16T22:34:12.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057700+0000", 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017350800000000001 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057874+0000", 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013362096 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071236+0000", 2020-11-16T22:34:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5955999999999998e-05 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071292+0000", 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5148000000000005e-05 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071377+0000", 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2495000000000003e-05 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071450+0000", 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66589 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3821537169999996, 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247, 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5683000000000003e-05 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962694+0000", 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2570000000000004e-06 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962700+0000", 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1571e-05 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962712+0000", 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.454e-06 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962716+0000", 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100496154 2020-11-16T22:34:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063212+0000", 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111053 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063324+0000", 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020392000000000001 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063527+0000", 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007972771 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071500+0000", 2020-11-16T22:34:12.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9499999999999996e-06 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071510+0000", 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8403000000000001e-05 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071589+0000", 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1031e-05 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071640+0000", 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.535 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66590 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3820434439999998, 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329, 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1128999999999999e-05 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962800+0000", 2020-11-16T22:34:12.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3040000000000001e-06 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962806+0000", 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1032e-05 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962817+0000", 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962821+0000", 2020-11-16T22:34:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104765866 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067587+0000", 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106117 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067693+0000", 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018714099999999999 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067880+0000", 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049221739999999996 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.072802+0000", 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043900800000000001 2020-11-16T22:34:12.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073241+0000", 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6645999999999999e-05 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073308+0000", 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.426e-05 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073352+0000", 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.546 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66591 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 6.3819393169999996, 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996, 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:12.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:34:12.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:12.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962907+0000", 2020-11-16T22:34:12.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0789999999999997e-06 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962913+0000", 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0987e-05 2020-11-16T22:34:12.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962924+0000", 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7900000000000001e-06 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962928+0000", 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062214039999999998 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969150+0000", 2020-11-16T22:34:12.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010196699999999999 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969252+0000", 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164537 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969416+0000", 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.089005452999999998 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:12.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058422+0000", 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003871518 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062293+0000", 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3857000000000003e-05 2020-11-16T22:34:12.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062347+0000", 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014316599999999999 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:12.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062490+0000", 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:12.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:12.558 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:12.559 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.573+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:12.559 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:12.660 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.674+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:12.710 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:68203 4.5 e193/117 4:b0fbeeaa:::1000000188d.00000016:head v 193'14315, mlcod=193'14308)", 2020-11-16T22:34:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:12.717516+0000", 2020-11-16T22:34:12.711 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.008314742, 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0083383269999999995, 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.712 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717516+0000", 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717516+0000", 2020-11-16T22:34:12.713 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5038999999999997e-05 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717601+0000", 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0037999999999999e-05 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.714 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717611+0000", 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5947000000000002e-05 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717627+0000", 2020-11-16T22:34:12.715 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.092e-06 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.717634+0000", 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0037691600000000001 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.716 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.721403+0000", 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3017999999999999e-05 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.721426+0000", 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039102939999999999 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.718 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:68206 3.2 3:545c15c7:::200.00000017:head [write 2068110~1927 [fadvise_dontneed] in=1927b] snapc 0=[] ondisk+write+known_if_redirected+full_force e193)", 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:12.723735+0000", 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.002094889, 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002188293, 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723735+0000", 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723735+0000", 2020-11-16T22:34:12.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5513999999999996e-05 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723811+0000", 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0266e-05 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723821+0000", 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7249999999999999e-05 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:12.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723838+0000", 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8010000000000004e-06 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723846+0000", 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6641999999999999e-05 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.723 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723883+0000", 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4207000000000002e-05 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.724 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.723967+0000", 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000109757 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:12.724077+0000", 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00034143699999999999 2020-11-16T22:34:12.725 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 2 2020-11-16T22:34:12.726 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:12.727 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:12.770 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:12.785+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:12.791 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:12.862 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:12.877+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:12.865 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:12.865 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:12.865 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:12.865 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:12.865 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:12.875 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:12.938 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:12.958 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:12.961 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:12.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:12.963 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:12.977+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.056 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.064 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.078+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.066 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:13.087 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:34:13.165 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.179+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.220 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.220 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.230 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:13.270 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.285+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.366 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.380+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.429 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.429 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.441 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:13.482 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.497+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.568 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:13.583+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:13.623 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.624 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.634 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:13.668 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.683+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.769 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.783+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:13.775 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.775 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.785 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:13.870 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:13.885+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:13.941 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:13.941 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:13.952 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:13.971 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:13.986+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.038 INFO:tasks.workunit.client.0.smithi138.stdout:40000 trunc from 0x2372fba to 0x3296874 2020-11-16T22:34:14.072 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.086+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.139 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:14.139 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:14.149 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:14.193 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.208+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.273 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:14.288+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:14.300 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:14.300 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:14.312 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:14.374 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:14.389+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:14.475 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.490+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.501 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:14.501 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:14.511 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:14.596 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.611+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.676 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:14.691+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:14.700 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:14.700 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:14.710 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:14.777 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.792+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.860 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:14.860 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:14.871 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:14.878 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.892+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:14.962 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:14.979 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:14.993+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:15.026 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:15.026 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:15.037 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:15.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~177s 2020-11-16T22:34:15.080 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:15.095+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:15.099 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:34:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:15.134 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:15.134 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:15.181 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:15.195+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:15.190 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:15.191 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:15.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:15.191 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:15.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 314.59035274899998, 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:15.192 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.193 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:15.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:15.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:15.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:15.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:15.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:15.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 256.29357767499999, 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:15.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:15.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:15.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:15.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.205 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:15.206 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.207 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.208 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:15.209 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.210 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:15.211 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.212 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 255.99520061600001, 2020-11-16T22:34:15.213 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:15.214 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.215 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.216 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:15.216 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:15.216 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:15.216 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.216 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:15.217 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:15.218 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:15.218 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.218 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.218 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:15.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:15.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:15.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:15.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.428 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.429 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 195.11825260200001, 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:15.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:15.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:15.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:15.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:15.634 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:15.634 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.634 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:15.635 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:15.637 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:15.638 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:15.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:15.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:15.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:16.046 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.046 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:16.046 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:24278 4.5 e142/117 4:a611fa68:::1000000188a.00000002:head v 142'2398, mlcod=142'2396)", 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 190.06565461599999, 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841, 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:16.047 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.048 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137869+0000", 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2353e-05 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137901+0000", 2020-11-16T22:34:16.049 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9810000000000003e-06 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137906+0000", 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4449999999999999e-06 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.050 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137914+0000", 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2179999999999999e-06 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137917+0000", 2020-11-16T22:34:16.051 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.2387000000000001e-05 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137939+0000", 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.26e-05 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.052 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.137952+0000", 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.043963884000000002 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:16.053 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.181916+0000", 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000176973 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.182093+0000", 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.044223841 2020-11-16T22:34:16.054 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:39959 4.1 4:8c3af0c3:::1000000188a.00000000:head [zero 3857876~38152] snapc 1=[] ondisk+write+known_if_redirected e142)", 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:16.055 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 190.06461128999999, 2020-11-16T22:34:16.056 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003, 2020-11-16T22:34:16.056 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:16.056 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:34:16.258 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:16.258 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 39959 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:16.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138912+0000", 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9079000000000001e-05 2020-11-16T22:34:16.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138931+0000", 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8539999999999999e-06 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138935+0000", 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6839999999999999e-06 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138942+0000", 2020-11-16T22:34:16.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9510000000000002e-06 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138945+0000", 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2508e-05 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.466 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:16.466 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.138987+0000", 2020-11-16T22:34:16.466 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.8102999999999997e-05 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.139036+0000", 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045085721000000002 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.467 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:05.184121+0000", 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045208900000000003 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.468 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.457815048000001, 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:16.469 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.470 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:16.471 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:16.471 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:16.471 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.471 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:16.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:16.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.676 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:16.677 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.678 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.679 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:16.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:16.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:16.885 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.457722502000003, 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:16.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:16.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:16.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:16.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:16.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:16.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:16.892 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:17.094 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:17.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:17.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:17.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:17.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.457317402000001, 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:17.098 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:17.099 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:17.099 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.099 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:17.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:17.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:17.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:17.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:17.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:17.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:17.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:17.307 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:17.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:17.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:17.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:17.512 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.457233438000003, 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.515 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:17.515 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:17.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:17.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.718 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.718 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:17.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:17.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.454823560000001, 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:17.724 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.725 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:17.726 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.727 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:17.728 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:17.728 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:17.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:17.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:17.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:17.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:17.934 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.452097258999999, 2020-11-16T22:34:17.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:17.935 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:17.935 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:17.935 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:18.138 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.138 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:18.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:18.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:18.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.142 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:18.143 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:18.143 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:18.143 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:18.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:18.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:18.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:18.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.348 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.450423436999998, 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:18.349 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:18.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:18.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:18.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.355 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:18.558 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:18.558 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:18.558 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.559 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:18.560 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:18.561 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:74463 4.4 e182/170 4:37415b28:::1000000188d.00000004:head v 182'4867, mlcod=182'4864)", 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.449683118999999, 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001, 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:18.562 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:18.563 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:18.563 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.563 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:18.563 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:18.766 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:18.766 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.766 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.766 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753841+0000", 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045129537999999997 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798970+0000", 2020-11-16T22:34:18.767 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8819999999999998e-06 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798976+0000", 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.3090000000000001e-06 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.768 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798985+0000", 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7199999999999997e-06 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798990+0000", 2020-11-16T22:34:18.769 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8662999999999999e-05 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799029+0000", 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7604999999999998e-05 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.770 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.771 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:18.771 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799046+0000", 2020-11-16T22:34:18.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0047328769999999999 2020-11-16T22:34:18.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803779+0000", 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.5707999999999997e-05 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:18.975 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803825+0000", 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.049984302000000001 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 52.014675963000002, 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:18.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:18.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:18.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:18.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:18.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:18.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:19.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:19.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:19.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:19.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:19.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:19.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:19.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:19.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:19.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:19.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.186 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:19.187 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:19.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:19.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:19.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:19.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:19.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:19.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:19.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:19.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:19.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:19.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:19.191 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.191 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:19.191 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.192 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.192 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:19.192 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:19.193 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.415944302, 2020-11-16T22:34:19.193 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:19.193 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:19.193 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:19.194 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:19.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.194 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:19.194 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:19.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:19.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.195 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:19.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:19.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:19.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:19.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:19.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:19.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:19.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:19.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:19.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:19.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:19.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:19.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:19.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:19.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:19.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:19.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:19.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:19.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.415870843, 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.612 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:19.613 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:19.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:19.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:19.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:19.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:19.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:19.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:19.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:19.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:19.824 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.415265017999999, 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:19.825 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:19.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:19.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:19.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:19.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:20.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:20.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:20.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:20.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:20.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:20.034 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:20.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:20.035 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.035 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:20.035 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.238 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:20.238 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:20.238 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 33.415107509000002, 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:20.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:20.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:20.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:20.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:20.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:20.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:20.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:20.449 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 9.2415570890000005, 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:20.450 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:20.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:20.654 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:20.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.654 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:20.655 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:20.656 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.657 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:20.658 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.659 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:20.660 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:20.661 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:20.662 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:20.663 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:20.664 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:20.664 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:20.666 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:34:20.667 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:20.667 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:20.679+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:20.871 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:20.885+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:20.872 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:20.873 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:20.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:20.972 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:20.986+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.017 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.017 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.027 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:21.073 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.087+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.173 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.188+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.186 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:21.274 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.289+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.328 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.328 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.338 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:21.375 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:21.390+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:21.476 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.491+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.501 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.501 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.512 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:21.577 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~183s 2020-11-16T22:34:21.578 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:21.592+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:21.679 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:21.693+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:21.707 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.707 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.718 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:21.779 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.794+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:21.867 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:21.867 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:21.878 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:21.880 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:21.894+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:21.981 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:21.995+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.043 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.043 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.053 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:22.082 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:22.096+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.182 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:22.197+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:22.201 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.201 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.211 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:22.283 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:22.298+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.384 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:22.399+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.394 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.394 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.407 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:22.485 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:22.500+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.558 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.558 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.572 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:22.587 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:22.601+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:22.687 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:22.702+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:22.757 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.758 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.770 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:22.813 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:22.828+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:22.871 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:34:22.877 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:22.889 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:22.903+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:22.920 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:22.920 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:22.931 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:22.989 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:23.004+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:23.067 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:23.081 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:23.081 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:23.088 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:23.088 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:23.088 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:23.088 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:23.088 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:23.090 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:23.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:23.092 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:23.092 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:23.105+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:23.191 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:23.206+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:23.273 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:23.274 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:23.274 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:23.274 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:23.274 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:23.284 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:23.326 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:23.340+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:23.393 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:23.407+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:23.486 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:23.486 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:23.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:23.486 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:23.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 265.49909831500003, 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:23.487 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:23.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:23.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:23.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:23.496 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 264.29094814799998, 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:23.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:23.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:23.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:23.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:23.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:23.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:23.710 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:23.710 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:23.710 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:23.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 200.95533055300001, 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:23.711 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:23.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:23.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:23.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:23.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:23.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:23.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:23.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:23.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:23.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:24.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 172.47451989999999, 2020-11-16T22:34:24.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:24.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:24.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:24.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:24.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:24.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 104.53508779800001, 2020-11-16T22:34:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.310173476000003, 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.310025093, 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66577 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 2104@3880336] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.537743407000001, 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634, 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6947000000000001e-05 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961450+0000", 2020-11-16T22:34:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7630000000000003e-06 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961456+0000", 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2191999999999999e-05 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961469+0000", 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961474+0000", 2020-11-16T22:34:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5272999999999999e-05 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961510+0000", 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100571 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961610+0000", 2020-11-16T22:34:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176787 2020-11-16T22:34:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961787+0000", 2020-11-16T22:34:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085900679999999993 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970377+0000", 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.091642535999999997 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062020+0000", 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0728000000000004e-05 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062100+0000", 2020-11-16T22:34:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2172000000000001e-05 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062152+0000", 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66578 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.537663733999999, 2020-11-16T22:34:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003, 2020-11-16T22:34:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:26.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:26.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3958e-05 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961526+0000", 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.014e-06 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:26.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961530+0000", 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7959999999999998e-06 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961537+0000", 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.897e-06 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961540+0000", 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5605000000000002e-05 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:26.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961576+0000", 2020-11-16T22:34:26.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5517000000000003e-05 2020-11-16T22:34:26.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:26.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961641+0000", 2020-11-16T22:34:26.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010893899999999999 2020-11-16T22:34:26.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961750+0000", 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073697459999999999 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969120+0000", 2020-11-16T22:34:26.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088765291999999996 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057885+0000", 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6371000000000004e-05 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057972+0000", 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013677599999999999 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058108+0000", 2020-11-16T22:34:26.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66579 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:26.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.53759732, 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999, 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:26.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6403000000000001e-05 2020-11-16T22:34:26.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961595+0000", 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9859999999999996e-06 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:26.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961599+0000", 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961606+0000", 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4480000000000001e-06 2020-11-16T22:34:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961609+0000", 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049254529999999998 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966534+0000", 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7503999999999997e-05 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:26.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966592+0000", 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7417999999999995e-05 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966689+0000", 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095498420000000001 2020-11-16T22:34:26.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062187+0000", 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4015000000000003e-05 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:26.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062251+0000", 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1647e-05 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062323+0000", 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2119e-05 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062375+0000", 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66580 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:26.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:26.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.537529746000001, 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688, 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:26.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4732999999999999e-05 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961661+0000", 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961665+0000", 2020-11-16T22:34:26.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9530000000000002e-06 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961672+0000", 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961682+0000", 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0902999999999998e-05 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961723+0000", 2020-11-16T22:34:26.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8277999999999999e-05 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961791+0000", 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010717000000000001 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961898+0000", 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10533300399999999 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067231+0000", 2020-11-16T22:34:26.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1401e-05 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067243+0000", 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1019999999999993e-05 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067324+0000", 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5339e-05 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067379+0000", 2020-11-16T22:34:26.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688 2020-11-16T22:34:26.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.854 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:26.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66581 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.53744708, 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006, 2020-11-16T22:34:26.855 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7623999999999998e-05 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:26.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961747+0000", 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961751+0000", 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0149999999999998e-06 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961758+0000", 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2649999999999998e-06 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:26.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961762+0000", 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4431000000000002e-05 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961807+0000", 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035119090000000001 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965319+0000", 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012228 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965441+0000", 2020-11-16T22:34:26.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092836962999999995 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058278+0000", 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013207889999999999 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059599+0000", 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2479999999999996e-05 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:26.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059681+0000", 2020-11-16T22:34:26.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5322999999999995e-05 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059776+0000", 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006 2020-11-16T22:34:27.066 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66582 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:27.067 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.537351297000001, 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302, 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:27.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0484e-05 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961865+0000", 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1609999999999997e-06 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961873+0000", 2020-11-16T22:34:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2751999999999999e-05 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961885+0000", 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4719999999999997e-06 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961890+0000", 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080703010000000002 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969960+0000", 2020-11-16T22:34:27.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7603999999999999e-05 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970018+0000", 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010071199999999999 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970118+0000", 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088442494999999996 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058561+0000", 2020-11-16T22:34:27.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037181409999999999 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062279+0000", 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3547000000000001e-05 2020-11-16T22:34:27.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062333+0000", 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3633000000000004e-05 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062416+0000", 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66583 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.537228728999999, 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072, 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:27.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6323999999999999e-05 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961984+0000", 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.109e-06 2020-11-16T22:34:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961991+0000", 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2452999999999999e-05 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962003+0000", 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962008+0000", 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046175440000000003 2020-11-16T22:34:27.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966625+0000", 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0504999999999998e-05 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:27.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966686+0000", 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3917000000000002e-05 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966780+0000", 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100608974 2020-11-16T22:34:27.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067389+0000", 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2806999999999999e-05 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:27.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067431+0000", 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6773999999999994e-05 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067508+0000", 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1255000000000003e-05 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067560+0000", 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66585 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.53699817, 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994, 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:27.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:27.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:27.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5401000000000001e-05 2020-11-16T22:34:27.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962213+0000", 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1690000000000003e-06 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962221+0000", 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2514000000000001e-05 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962233+0000", 2020-11-16T22:34:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.378e-06 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962238+0000", 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032176309999999999 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965456+0000", 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101195 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965557+0000", 2020-11-16T22:34:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149464 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965707+0000", 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092509202999999998 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058216+0000", 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013792990000000001 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059595+0000", 2020-11-16T22:34:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4564e-05 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059670+0000", 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013108299999999999 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059801+0000", 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66586 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.536878615999999, 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146, 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5766999999999997e-05 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:27.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962333+0000", 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4960000000000001e-06 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962340+0000", 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.313e-05 2020-11-16T22:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962353+0000", 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.763e-06 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:27.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962358+0000", 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080385160000000008 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970396+0000", 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.08724875 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057645+0000", 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018116900000000001 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:27.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057826+0000", 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063036259999999997 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064130+0000", 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030772299999999998 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064438+0000", 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5346000000000003e-05 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:27.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064483+0000", 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8860000000000002e-05 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064512+0000", 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.713 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66587 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.536763604000001, 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938, 2020-11-16T22:34:27.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:27.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:27.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6044000000000002e-05 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962449+0000", 2020-11-16T22:34:27.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9759999999999998e-06 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962456+0000", 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2969999999999999e-05 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962469+0000", 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.977e-06 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962474+0000", 2020-11-16T22:34:27.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079504459999999999 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970424+0000", 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.087276322000000003 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:27.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:27.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:27.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057700+0000", 2020-11-16T22:34:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017350800000000001 2020-11-16T22:34:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057874+0000", 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013362096 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071236+0000", 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5955999999999998e-05 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071292+0000", 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5148000000000005e-05 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071377+0000", 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2495000000000003e-05 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071450+0000", 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66589 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.536517830000001, 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247, 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5683000000000003e-05 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962694+0000", 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2570000000000004e-06 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962700+0000", 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1571e-05 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962712+0000", 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.454e-06 2020-11-16T22:34:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962716+0000", 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100496154 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063212+0000", 2020-11-16T22:34:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111053 2020-11-16T22:34:28.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:28.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063324+0000", 2020-11-16T22:34:28.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020392000000000001 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063527+0000", 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007972771 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071500+0000", 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9499999999999996e-06 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071510+0000", 2020-11-16T22:34:28.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8403000000000001e-05 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071589+0000", 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1031e-05 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071640+0000", 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.343 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66590 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.536407557, 2020-11-16T22:34:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329, 2020-11-16T22:34:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1128999999999999e-05 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962800+0000", 2020-11-16T22:34:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3040000000000001e-06 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962806+0000", 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1032e-05 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962817+0000", 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962821+0000", 2020-11-16T22:34:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104765866 2020-11-16T22:34:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:28.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067587+0000", 2020-11-16T22:34:28.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106117 2020-11-16T22:34:28.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067693+0000", 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018714099999999999 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067880+0000", 2020-11-16T22:34:28.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049221739999999996 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.072802+0000", 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043900800000000001 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073241+0000", 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6645999999999999e-05 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073308+0000", 2020-11-16T22:34:28.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.426e-05 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073352+0000", 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:28.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66591 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.53630343, 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996, 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962907+0000", 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0789999999999997e-06 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962913+0000", 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0987e-05 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962924+0000", 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7900000000000001e-06 2020-11-16T22:34:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962928+0000", 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062214039999999998 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969150+0000", 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010196699999999999 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969252+0000", 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164537 2020-11-16T22:34:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969416+0000", 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.089005452999999998 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058422+0000", 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003871518 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062293+0000", 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3857000000000003e-05 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062347+0000", 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014316599999999999 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062490+0000", 2020-11-16T22:34:29.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996 2020-11-16T22:34:29.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:29.174 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:29.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:29.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:29.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:29.175 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:29.176 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:34:29.178 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:29.190+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:29.178 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:29.278 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:29.290+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:29.328 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:72365 3.2 3:590b566d:::200.00000018:head [write 91215~1932 [fadvise_dontneed] in=1932b] snapc 0=[] ondisk+write+known_if_redirected+full_force e196)", 2020-11-16T22:34:29.328 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:29.342868+0000", 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.000427501, 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000445752, 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:29.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342868+0000", 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342868+0000", 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5098000000000002e-05 2020-11-16T22:34:29.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342903+0000", 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3399999999999997e-06 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342909+0000", 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2113000000000001e-05 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342921+0000", 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3959999999999997e-06 2020-11-16T22:34:29.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342926+0000", 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7827999999999998e-05 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:29.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.342964+0000", 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.4028000000000004e-05 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.343028+0000", 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.6501000000000003e-05 2020-11-16T22:34:29.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:29.343124+0000", 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000256304 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:29.335 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:29.538 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:29.538 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:34:29.538 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:34:29.538 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:29.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:29.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:29.541 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:29.542 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:29.553+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:29.640 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:29.655+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:29.722 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:29.722 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:29.732 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:29.776 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:29.790+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:29.841 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:29.856+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:29.877 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:29.877 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:29.889 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:29.942 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:29.956+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:30.042 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:30.043 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:30.043 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.057+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.054 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:30.144 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.158+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~192s 2020-11-16T22:34:30.244 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.259+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.245 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:30.245 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:30.256 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:30.345 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.360+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.446 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.460+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.447 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:30.447 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:30.458 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:30.547 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:30.560+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:30.613 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:30.614 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:30.625 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:30.647 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.662+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:30.748 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:30.762+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:30.842 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:30.843 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:30.849 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:30.863+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:30.853 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:30.949 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:30.964+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:31.038 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:31.039 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:31.050 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:31.051 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:31.064+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:31.150 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.164+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.179 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:34:31.214 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:31.215 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:31.227 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:31.251 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.265+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.351 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:31.366+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:31.380 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:31.381 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:31.392 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:31.452 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.466+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.542 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:31.553 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.566+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.572 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:31.572 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:31.585 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:31.682 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:31.697+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:31.689 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:31.712 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:31.712 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:31.712 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:31.712 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:31.712 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:31.714 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:31.714 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:31.782 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:31.797+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:73007 4.5 e196/117 4:ab344873:::1000000188d.00000010:head v 196'15689, mlcod=196'15685)", 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:31.838220+0000", 2020-11-16T22:34:31.830 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0069195350000000001, 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.006941633, 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "started", 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:31.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838220+0000", 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838220+0000", 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9507e-05 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838260+0000", 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6699999999999999e-06 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:31.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838266+0000", 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.8400000000000001e-06 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838274+0000", 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7409999999999998e-06 2020-11-16T22:34:31.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838280+0000", 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00025306999999999999 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:31.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838533+0000", 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.7427000000000001e-05 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:31.838551+0000", 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000330255 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:31.836 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:31.837 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:31.837 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:31.837 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:34:31.837 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:34:31.837 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:31.840 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:31.883 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.897+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.984 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:31.997+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:31.992 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:31.992 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:31.992 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:31.992 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:31.993 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:32.004 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:32.088 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:32.102+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:32.131 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:34:32.185 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:32.200+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:32.194 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:32.194 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:32.194 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:32.194 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:32.194 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 331.59433247099997, 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:32.195 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:32.196 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:32.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:32.198 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:32.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:32.200 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:32.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:32.202 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.203 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.204 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 273.29755739699999, 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:32.205 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:32.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:32.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:32.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 272.99918033799997, 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:32.415 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:32.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.618 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:32.618 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:32.618 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:32.618 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.619 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:32.620 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.621 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:32.622 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:32.623 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.623 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.623 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:32.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:32.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:32.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:32.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:32.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:32.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:32.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:32.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 212.12223232400001, 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:32.832 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.833 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:32.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:32.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:32.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.038 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:33.039 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.040 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:33.041 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:33.042 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.461794769999997, 2020-11-16T22:34:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:33.043 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:33.246 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:33.246 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:33.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:33.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:33.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:33.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:33.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:33.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:33.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.253 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:33.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.257 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:33.458 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:33.458 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.461702224000007, 2020-11-16T22:34:33.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:33.458 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:33.458 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:33.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:33.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:33.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:33.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:33.463 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.666 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.666 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:33.666 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:33.666 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:33.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.667 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:33.668 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.669 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:33.670 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.671 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.461297123999998, 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:33.672 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:33.673 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:33.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.675 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:33.676 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:33.676 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:33.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.676 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:33.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:33.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:33.881 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.46121316, 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:33.882 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:33.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.086 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.087 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:34.088 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.089 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:34.090 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.091 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:34.092 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:34.093 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.458803282000005, 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.094 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:34.095 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:34.096 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:34.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:34.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.298 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:34.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:34.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:34.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:34.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:34.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.506 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:34.506 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.456076980999995, 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:34.507 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.508 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:34.509 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.510 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:34.511 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.512 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:34.513 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:34.514 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.454403158999995, 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:34.515 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:34.718 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:34.718 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:34.718 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:34.718 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:34.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.719 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:34.720 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.721 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:34.722 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:34.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:34.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:34.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:34.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:34.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:34.931 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 69.018655684999999, 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:34.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:34.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:34.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:34.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.138 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:35.138 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:35.138 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:35.139 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.141 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.419924023999997, 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.142 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.143 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.143 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.143 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.346 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:35.346 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.346 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.347 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.348 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.349 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.350 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:35.351 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.352 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.353 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.354 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.355 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.355 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.355 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:35.355 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.419850564999997, 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.356 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:35.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.358 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.358 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.358 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:35.359 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.360 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:35.361 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.362 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:35.363 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.364 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:35.366 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.419244740000003, 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.367 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.368 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.369 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.370 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:35.373 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:35.374 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 50.419087230999999, 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:35.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:35.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:35.382 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 26.245536811000001, 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:35.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:35.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:35.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:69916 3.2 3:545c15c7:::200.00000017:head [write 2999902~1932 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e195)", 2020-11-16T22:34:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 12.588964858000001, 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007, 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7635999999999997e-05 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618576+0000", 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618583+0000", 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0594000000000001e-05 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618593+0000", 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2549999999999999e-06 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618598+0000", 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6642999999999999e-05 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618624+0000", 2020-11-16T22:34:35.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9358000000000003e-05 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618684+0000", 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4988000000000006e-05 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618779+0000", 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035862419999999999 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.622365+0000", 2020-11-16T22:34:35.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.061623445999999998 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.683988+0000", 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7128e-05 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684035+0000", 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8116e-05 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684064+0000", 2020-11-16T22:34:35.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87626 4.6 e195/184 4:6090c0e3:::1000000188d.00000003:head v 195'3820, mlcod=195'3818)", 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:35.402 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.575177015, 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874, 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:35.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000127615 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632454+0000", 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010485899999999999 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632559+0000", 2020-11-16T22:34:35.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9750000000000007e-06 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632567+0000", 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7689999999999998e-06 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632571+0000", 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0516000000000003e-05 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632611+0000", 2020-11-16T22:34:35.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1466000000000001e-05 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632633+0000", 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.076359815999999997 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.708992+0000", 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2724000000000004e-05 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.709065+0000", 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874 2020-11-16T22:34:35.409 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87627 4.4 e195/170 4:218b21fa:::1000000188d.00000001:head v 195'7868, mlcod=195'7866)", 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:35.410 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 11.574890004, 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005, 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:35.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9462000000000006e-05 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632693+0000", 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0729999999999999e-06 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632700+0000", 2020-11-16T22:34:35.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0679999999999999e-05 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632711+0000", 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3640000000000001e-06 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632715+0000", 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9465000000000002e-05 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632764+0000", 2020-11-16T22:34:35.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9893000000000001e-05 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632784+0000", 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078505851000000001 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711290+0000", 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3350000000000003e-05 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711344+0000", 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005 2020-11-16T22:34:35.417 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.418 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.418 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.418 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:35.418 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:35.418 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:35.419 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:35.419 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:34:35.421 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:35.422 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:35.434+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:35.520 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:35.535+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:90585 4.5 4:b0fbeeaa:::1000000188d.00000016:head [write 1200128~29173 [2593@0] in=29173b] snapc 1=[] ondisk+write+known_if_redirected e197)", 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:35.584367+0000", 2020-11-16T22:34:35.574 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0047400189999999998, 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047611570000000002, 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:35.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 90585 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584367+0000", 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584367+0000", 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3889999999999997e-05 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:35.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584461+0000", 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6819999999999998e-06 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584470+0000", 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7646999999999999e-05 2020-11-16T22:34:35.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584488+0000", 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7919999999999993e-06 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584496+0000", 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3624000000000002e-05 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584530+0000", 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1173000000000005e-05 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584591+0000", 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000145809 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:35.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584736+0000", 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036961699999999998 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:90586 4.4 4:218b21fa:::1000000188d.00000001:head [write 3881608~32295 [2593@4194304] in=32295b] snapc 1=[] ondisk+write+known_if_redirected e197)", 2020-11-16T22:34:35.582 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:35.584549+0000", 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0045576530000000001, 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046339909999999996, 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:35.583 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 90586 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584549+0000", 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:35.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584549+0000", 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4438999999999997e-05 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584624+0000", 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0679999999999999e-06 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584631+0000", 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7543000000000001e-05 2020-11-16T22:34:35.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584648+0000", 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9809999999999997e-06 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584655+0000", 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2534000000000002e-05 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584708+0000", 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101778 2020-11-16T22:34:35.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584810+0000", 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017476899999999999 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584984+0000", 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043511199999999998 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:90587 4.4 4.4538e79c (undecoded) ondisk+write+known_if_redirected e197)", 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:35.584713+0000", 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0043942759999999999, 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045147479999999999, 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:35.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 90587 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584713+0000", 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:35.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584713+0000", 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010068699999999999 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584813+0000", 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5830000000000001e-06 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584821+0000", 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3906e-05 2020-11-16T22:34:35.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584835+0000", 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4609999999999999e-06 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:35.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:35.584840+0000", 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000127637 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:34:35.596 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 3 2020-11-16T22:34:35.597 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:35.597 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:35.598 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:35.599 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:35.600 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:35.600 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:35.621 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:35.636+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:35.721 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:35.736+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:35.743 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:35.743 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:35.756 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:35.823 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:35.837+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:35.905 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:35.905 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:35.915 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:35.958 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:35.973+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.024 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:36.039+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:36.063 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:36.063 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:36.076 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:34:36.159 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.174+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.226 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.241+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~198s 2020-11-16T22:34:36.265 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:36.265 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:36.276 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:36.360 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.375+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.428 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.443+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.461 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:36.462 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:36.475 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:36.529 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:36.544+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:36.630 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:36.630 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:36.630 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.645+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:36.639 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:36.730 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:36.745+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:36.831 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:36.845+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:36.837 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:36.931 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:36.946+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.032 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.047+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.041 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:37.041 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:37.051 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:37.133 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.148+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.190 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:37.191 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:37.200 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:37.234 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:37.249+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:37.335 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:37.350+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:37.393 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:37.393 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:37.402 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:37.437 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:37.451+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:37.538 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.553+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.560 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:37.561 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:37.573 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:37.601 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:37.639 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.654+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.716 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:37.717 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:37.726 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:37.740 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.755+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.747 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:37.768 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:37.768 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:37.769 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:37.769 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:37.769 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:37.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:37.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:37.840 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:37.855+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:37.877 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:37.877 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:37.887 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:37.941 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:37.955+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:38.041 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:38.056+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:38.044 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:38.045 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 280.05772945400003, 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:38.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:38.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:38.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 278.84957928699998, 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.055 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:38.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:38.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:38.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:38.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 215.51396169200001, 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:38.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:38.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:38.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 187.03315103899999, 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:38.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:38.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:38.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:38.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 119.09371893700001, 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:38.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:38.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:38.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.102 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.868804615000002, 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:38.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:38.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:38.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:38.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:38.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.868656232000006, 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.112 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:38.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:38.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66577 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 2104@3880336] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.096374546, 2020-11-16T22:34:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634, 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6947000000000001e-05 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961450+0000", 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7630000000000003e-06 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961456+0000", 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2191999999999999e-05 2020-11-16T22:34:38.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961469+0000", 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961474+0000", 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5272999999999999e-05 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961510+0000", 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100571 2020-11-16T22:34:38.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961610+0000", 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176787 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961787+0000", 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085900679999999993 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970377+0000", 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.091642535999999997 2020-11-16T22:34:38.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062020+0000", 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0728000000000004e-05 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062100+0000", 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2172000000000001e-05 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062152+0000", 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66578 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.096294872999998, 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003, 2020-11-16T22:34:38.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3958e-05 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961526+0000", 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.014e-06 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961530+0000", 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7959999999999998e-06 2020-11-16T22:34:38.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961537+0000", 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.897e-06 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961540+0000", 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5605000000000002e-05 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961576+0000", 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5517000000000003e-05 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961641+0000", 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010893899999999999 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961750+0000", 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073697459999999999 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969120+0000", 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088765291999999996 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057885+0000", 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6371000000000004e-05 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057972+0000", 2020-11-16T22:34:38.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013677599999999999 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058108+0000", 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66579 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.096228459000002, 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999, 2020-11-16T22:34:38.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6403000000000001e-05 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961595+0000", 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9859999999999996e-06 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961599+0000", 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2020-11-16T22:34:38.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961606+0000", 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4480000000000001e-06 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961609+0000", 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049254529999999998 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966534+0000", 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7503999999999997e-05 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966592+0000", 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7417999999999995e-05 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966689+0000", 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095498420000000001 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062187+0000", 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4015000000000003e-05 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062251+0000", 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1647e-05 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062323+0000", 2020-11-16T22:34:38.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2119e-05 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062375+0000", 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.152 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66580 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.096160885000003, 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688, 2020-11-16T22:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4732999999999999e-05 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961661+0000", 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961665+0000", 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9530000000000002e-06 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961672+0000", 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961682+0000", 2020-11-16T22:34:38.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0902999999999998e-05 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961723+0000", 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8277999999999999e-05 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961791+0000", 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010717000000000001 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961898+0000", 2020-11-16T22:34:38.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10533300399999999 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067231+0000", 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1401e-05 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067243+0000", 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1019999999999993e-05 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067324+0000", 2020-11-16T22:34:38.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5339e-05 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067379+0000", 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.162 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66581 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.096078218999999, 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006, 2020-11-16T22:34:38.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7623999999999998e-05 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961747+0000", 2020-11-16T22:34:38.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961751+0000", 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0149999999999998e-06 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961758+0000", 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2649999999999998e-06 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961762+0000", 2020-11-16T22:34:38.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4431000000000002e-05 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961807+0000", 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035119090000000001 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965319+0000", 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012228 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965441+0000", 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092836962999999995 2020-11-16T22:34:38.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058278+0000", 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013207889999999999 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059599+0000", 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2479999999999996e-05 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059681+0000", 2020-11-16T22:34:38.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5322999999999995e-05 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059776+0000", 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66582 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095982436, 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302, 2020-11-16T22:34:38.173 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0484e-05 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961865+0000", 2020-11-16T22:34:38.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1609999999999997e-06 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961873+0000", 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2751999999999999e-05 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961885+0000", 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4719999999999997e-06 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961890+0000", 2020-11-16T22:34:38.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080703010000000002 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969960+0000", 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7603999999999999e-05 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970018+0000", 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010071199999999999 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970118+0000", 2020-11-16T22:34:38.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088442494999999996 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058561+0000", 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037181409999999999 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062279+0000", 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3547000000000001e-05 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062333+0000", 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3633000000000004e-05 2020-11-16T22:34:38.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062416+0000", 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66583 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095859867999998, 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072, 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6323999999999999e-05 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961984+0000", 2020-11-16T22:34:38.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.109e-06 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961991+0000", 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2452999999999999e-05 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962003+0000", 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962008+0000", 2020-11-16T22:34:38.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046175440000000003 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966625+0000", 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0504999999999998e-05 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966686+0000", 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3917000000000002e-05 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966780+0000", 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100608974 2020-11-16T22:34:38.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067389+0000", 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2806999999999999e-05 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067431+0000", 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6773999999999994e-05 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067508+0000", 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1255000000000003e-05 2020-11-16T22:34:38.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067560+0000", 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66585 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095629309000003, 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994, 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.193 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5401000000000001e-05 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962213+0000", 2020-11-16T22:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1690000000000003e-06 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962221+0000", 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2514000000000001e-05 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962233+0000", 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.378e-06 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962238+0000", 2020-11-16T22:34:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032176309999999999 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965456+0000", 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101195 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965557+0000", 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149464 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965707+0000", 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092509202999999998 2020-11-16T22:34:38.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058216+0000", 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013792990000000001 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059595+0000", 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4564e-05 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059670+0000", 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013108299999999999 2020-11-16T22:34:38.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059801+0000", 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.202 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66586 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095509755000002, 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146, 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.203 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5766999999999997e-05 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962333+0000", 2020-11-16T22:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4960000000000001e-06 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962340+0000", 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.313e-05 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962353+0000", 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.763e-06 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962358+0000", 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080385160000000008 2020-11-16T22:34:38.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970396+0000", 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.08724875 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057645+0000", 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018116900000000001 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057826+0000", 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063036259999999997 2020-11-16T22:34:38.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064130+0000", 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030772299999999998 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064438+0000", 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5346000000000003e-05 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064483+0000", 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8860000000000002e-05 2020-11-16T22:34:38.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064512+0000", 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66587 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095394743, 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938, 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6044000000000002e-05 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962449+0000", 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9759999999999998e-06 2020-11-16T22:34:38.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962456+0000", 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2969999999999999e-05 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962469+0000", 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.977e-06 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962474+0000", 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079504459999999999 2020-11-16T22:34:38.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970424+0000", 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.087276322000000003 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057700+0000", 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017350800000000001 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057874+0000", 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013362096 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071236+0000", 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5955999999999998e-05 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071292+0000", 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5148000000000005e-05 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071377+0000", 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2495000000000003e-05 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071450+0000", 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66589 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095148969, 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247, 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.223 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5683000000000003e-05 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962694+0000", 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2570000000000004e-06 2020-11-16T22:34:38.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962700+0000", 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1571e-05 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962712+0000", 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.454e-06 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962716+0000", 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100496154 2020-11-16T22:34:38.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063212+0000", 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111053 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063324+0000", 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020392000000000001 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063527+0000", 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007972771 2020-11-16T22:34:38.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071500+0000", 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9499999999999996e-06 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071510+0000", 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8403000000000001e-05 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071589+0000", 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1031e-05 2020-11-16T22:34:38.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071640+0000", 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66590 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.095038696000003, 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329, 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.233 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1128999999999999e-05 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962800+0000", 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3040000000000001e-06 2020-11-16T22:34:38.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962806+0000", 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1032e-05 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962817+0000", 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962821+0000", 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104765866 2020-11-16T22:34:38.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067587+0000", 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106117 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067693+0000", 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018714099999999999 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067880+0000", 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049221739999999996 2020-11-16T22:34:38.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.072802+0000", 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043900800000000001 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073241+0000", 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6645999999999999e-05 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073308+0000", 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.426e-05 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073352+0000", 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66591 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.094934569000003, 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996, 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.243 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:38.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962907+0000", 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0789999999999997e-06 2020-11-16T22:34:38.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962913+0000", 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0987e-05 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962924+0000", 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7900000000000001e-06 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962928+0000", 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062214039999999998 2020-11-16T22:34:38.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969150+0000", 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010196699999999999 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969252+0000", 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164537 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969416+0000", 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.089005452999999998 2020-11-16T22:34:38.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058422+0000", 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003871518 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:38.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062293+0000", 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3857000000000003e-05 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062347+0000", 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014316599999999999 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:38.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062490+0000", 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:38.253 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:38.253 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:38.253 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T22:34:38.255 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:38.268+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:38.255 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:38.354 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:38.369+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:38.407 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:74500 3.2 3:590b566d:::200.00000018:head [write 1246412~1932 [fadvise_dontneed] in=1932b] snapc 0=[] ondisk+write+known_if_redirected+full_force e198)", 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:38.422087+0000", 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.00099662699999999993, 2020-11-16T22:34:38.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0010171679999999999, 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422087+0000", 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422087+0000", 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5537000000000002e-05 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422122+0000", 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1979999999999998e-06 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422128+0000", 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1221e-05 2020-11-16T22:34:38.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422139+0000", 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6439999999999997e-06 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.412 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422145+0000", 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5562999999999998e-05 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422181+0000", 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.9275000000000001e-05 2020-11-16T22:34:38.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422280+0000", 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014871 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:38.422429+0000", 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00034214800000000002 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:34:38.415 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:38.418 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:38.461 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:38.476+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:38.556 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:38.571+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:38.573 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:38.574 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:38.574 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:38.586 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:38.669 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:38.684+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:38.757 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:38.772+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:38.801 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:38.801 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 338.20196179200002, 2020-11-16T22:34:38.802 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:38.803 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.804 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:38.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.806 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:38.807 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.808 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:38.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.810 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:38.811 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 279.90518671799998, 2020-11-16T22:34:38.812 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:38.813 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.814 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:38.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.816 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:38.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:38.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:38.821 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 279.60680965900002, 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.822 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:38.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.824 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:38.825 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.826 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:38.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.828 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.829 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.830 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.831 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.832 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 218.729861645, 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.833 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.834 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.835 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.836 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.837 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.838 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.839 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.840 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.069424091000002, 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.841 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.842 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.843 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:38.844 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.845 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:38.846 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.847 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:38.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.850 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.069331544999997, 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:38.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:38.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.068926445000002, 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:38.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:38.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.867 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.868 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.068842481000004, 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:38.870 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.871 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:38.872 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.873 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:38.874 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.875 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.876 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.066432602999996, 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.877 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:38.878 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.879 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:38.880 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.881 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:38.882 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.883 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.884 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.063706302, 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.885 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:38.886 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.887 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:38.888 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.889 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:38.890 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.891 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.892 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 76.062032479999999, 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:38.893 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:38.894 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.895 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:38.896 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.897 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.899 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:38.900 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.901 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:38.902 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 75.626285006000003, 2020-11-16T22:34:38.903 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.904 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.905 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:38.906 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.907 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:38.908 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.909 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.910 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.027553345000001, 2020-11-16T22:34:38.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:38.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.915 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.916 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.917 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.918 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:38.919 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.920 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.027479886000002, 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:38.921 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.922 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.923 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.924 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:38.925 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.926 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.928 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:38.929 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.930 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.026874061000001, 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.931 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.932 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:38.933 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.934 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:38.935 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.936 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:38.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.938 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 57.026716552000003, 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.939 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.940 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:38.941 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.942 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:38.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.944 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:38.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.946 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 32.853166131999998, 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.947 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.948 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:38.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.950 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:38.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.952 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:38.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:38.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:69916 3.2 3:545c15c7:::200.00000017:head [write 2999902~1932 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e195)", 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 19.196594179000002, 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007, 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.957 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.958 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7635999999999997e-05 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618576+0000", 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:34:38.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618583+0000", 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0594000000000001e-05 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.960 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618593+0000", 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2549999999999999e-06 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618598+0000", 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6642999999999999e-05 2020-11-16T22:34:38.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618624+0000", 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9358000000000003e-05 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.962 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618684+0000", 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4988000000000006e-05 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618779+0000", 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035862419999999999 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.963 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.622365+0000", 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.061623445999999998 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.683988+0000", 2020-11-16T22:34:38.964 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7128e-05 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684035+0000", 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8116e-05 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.965 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684064+0000", 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.966 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87626 4.6 e195/184 4:6090c0e3:::1000000188d.00000003:head v 195'3820, mlcod=195'3818)", 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 18.182806335999999, 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874, 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.967 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.968 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000127615 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632454+0000", 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010485899999999999 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.969 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632559+0000", 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9750000000000007e-06 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632567+0000", 2020-11-16T22:34:38.970 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7689999999999998e-06 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632571+0000", 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0516000000000003e-05 2020-11-16T22:34:38.971 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.972 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.972 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.972 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632611+0000", 2020-11-16T22:34:38.972 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1466000000000001e-05 2020-11-16T22:34:38.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632633+0000", 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.076359815999999997 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.708992+0000", 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2724000000000004e-05 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.709065+0000", 2020-11-16T22:34:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87627 4.4 e195/170 4:218b21fa:::1000000188d.00000001:head v 195'7868, mlcod=195'7866)", 2020-11-16T22:34:38.975 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 18.182519325000001, 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005, 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:38.976 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9462000000000006e-05 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.977 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632693+0000", 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0729999999999999e-06 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:38.978 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632700+0000", 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0679999999999999e-05 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632711+0000", 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3640000000000001e-06 2020-11-16T22:34:38.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632715+0000", 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9465000000000002e-05 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.980 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632764+0000", 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9893000000000001e-05 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632784+0000", 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078505851000000001 2020-11-16T22:34:38.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711290+0000", 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3350000000000003e-05 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:38.982 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711344+0000", 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:38.983 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:38.985 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:38.999+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:38.985 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:39.085 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:39.100+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:39.147 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:39.147 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:91312 4.1 4:98a9bd97:::1000000188d.0000000a:head [zero 1329718~64961] snapc 1=[] ondisk+write+known_if_redirected e198)", 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:39.158711+0000", 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0040642159999999998, 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040897709999999999, 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 91312 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158711+0000", 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158711+0000", 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3591000000000002e-05 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158765+0000", 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3469999999999999e-06 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158773+0000", 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4606999999999999e-05 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158788+0000", 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8990000000000001e-06 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158796+0000", 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8149000000000003e-05 2020-11-16T22:34:39.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158844+0000", 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011113800000000001 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:39.158955+0000", 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024373099999999999 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:34:39.155 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:34:39.156 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:39.156 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is set 2020-11-16T22:34:39.157 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:39.186 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:39.201+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:39.287 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:39.301+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:39.312 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:39.313 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:39.313 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:39.313 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:39.313 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:39.325 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:39.408 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:39.423+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:39.488 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:39.502+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:39.509 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 281.52270641400003, 2020-11-16T22:34:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 280.31455624699998, 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 216.97893865200001, 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 188.49812799899999, 2020-11-16T22:34:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.55869589700001, 2020-11-16T22:34:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.333781575000003, 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.333633191999994, 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66577 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 2104@3880336] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.561351506000001, 2020-11-16T22:34:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634, 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6947000000000001e-05 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961450+0000", 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7630000000000003e-06 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961456+0000", 2020-11-16T22:34:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2191999999999999e-05 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961469+0000", 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961474+0000", 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5272999999999999e-05 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961510+0000", 2020-11-16T22:34:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100571 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961610+0000", 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176787 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961787+0000", 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085900679999999993 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970377+0000", 2020-11-16T22:34:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.091642535999999997 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062020+0000", 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0728000000000004e-05 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062100+0000", 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2172000000000001e-05 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062152+0000", 2020-11-16T22:34:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66578 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.561271832999999, 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003, 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3958e-05 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961526+0000", 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.014e-06 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961530+0000", 2020-11-16T22:34:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7959999999999998e-06 2020-11-16T22:34:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961537+0000", 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.897e-06 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961540+0000", 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5605000000000002e-05 2020-11-16T22:34:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961576+0000", 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5517000000000003e-05 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961641+0000", 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010893899999999999 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961750+0000", 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073697459999999999 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969120+0000", 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088765291999999996 2020-11-16T22:34:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057885+0000", 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6371000000000004e-05 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057972+0000", 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013677599999999999 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058108+0000", 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003 2020-11-16T22:34:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66579 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.561205418999997, 2020-11-16T22:34:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999, 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6403000000000001e-05 2020-11-16T22:34:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961595+0000", 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9859999999999996e-06 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961599+0000", 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961606+0000", 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4480000000000001e-06 2020-11-16T22:34:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961609+0000", 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049254529999999998 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966534+0000", 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7503999999999997e-05 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966592+0000", 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7417999999999995e-05 2020-11-16T22:34:39.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966689+0000", 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095498420000000001 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062187+0000", 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4015000000000003e-05 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062251+0000", 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1647e-05 2020-11-16T22:34:39.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062323+0000", 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2119e-05 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062375+0000", 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66580 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.561137844999998, 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688, 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:39.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4732999999999999e-05 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961661+0000", 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961665+0000", 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9530000000000002e-06 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961672+0000", 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961682+0000", 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0902999999999998e-05 2020-11-16T22:34:39.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961723+0000", 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8277999999999999e-05 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961791+0000", 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010717000000000001 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961898+0000", 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10533300399999999 2020-11-16T22:34:39.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067231+0000", 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1401e-05 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067243+0000", 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1019999999999993e-05 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067324+0000", 2020-11-16T22:34:39.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5339e-05 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067379+0000", 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.630 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66581 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.630 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:39.630 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.561055179, 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006, 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7623999999999998e-05 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961747+0000", 2020-11-16T22:34:39.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961751+0000", 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0149999999999998e-06 2020-11-16T22:34:39.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961758+0000", 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2649999999999998e-06 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961762+0000", 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4431000000000002e-05 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961807+0000", 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035119090000000001 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965319+0000", 2020-11-16T22:34:39.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012228 2020-11-16T22:34:39.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965441+0000", 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092836962999999995 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058278+0000", 2020-11-16T22:34:39.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013207889999999999 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059599+0000", 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2479999999999996e-05 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059681+0000", 2020-11-16T22:34:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5322999999999995e-05 2020-11-16T22:34:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059776+0000", 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.643 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66582 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.643 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:39.643 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560959396000001, 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302, 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:39.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:39.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0484e-05 2020-11-16T22:34:39.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961865+0000", 2020-11-16T22:34:39.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1609999999999997e-06 2020-11-16T22:34:39.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961873+0000", 2020-11-16T22:34:39.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2751999999999999e-05 2020-11-16T22:34:39.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961885+0000", 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4719999999999997e-06 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961890+0000", 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080703010000000002 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969960+0000", 2020-11-16T22:34:39.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7603999999999999e-05 2020-11-16T22:34:39.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970018+0000", 2020-11-16T22:34:39.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010071199999999999 2020-11-16T22:34:39.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970118+0000", 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088442494999999996 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058561+0000", 2020-11-16T22:34:39.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037181409999999999 2020-11-16T22:34:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062279+0000", 2020-11-16T22:34:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3547000000000001e-05 2020-11-16T22:34:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062333+0000", 2020-11-16T22:34:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3633000000000004e-05 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062416+0000", 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66583 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560836827999999, 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072, 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6323999999999999e-05 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961984+0000", 2020-11-16T22:34:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.109e-06 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961991+0000", 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2452999999999999e-05 2020-11-16T22:34:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962003+0000", 2020-11-16T22:34:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962008+0000", 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046175440000000003 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966625+0000", 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0504999999999998e-05 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966686+0000", 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3917000000000002e-05 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966780+0000", 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100608974 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067389+0000", 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2806999999999999e-05 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067431+0000", 2020-11-16T22:34:39.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6773999999999994e-05 2020-11-16T22:34:39.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067508+0000", 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1255000000000003e-05 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067560+0000", 2020-11-16T22:34:39.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66585 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.668 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560606268999997, 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994, 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5401000000000001e-05 2020-11-16T22:34:39.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962213+0000", 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1690000000000003e-06 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962221+0000", 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2514000000000001e-05 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962233+0000", 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.378e-06 2020-11-16T22:34:39.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962238+0000", 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032176309999999999 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965456+0000", 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101195 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965557+0000", 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149464 2020-11-16T22:34:39.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965707+0000", 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092509202999999998 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058216+0000", 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013792990000000001 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059595+0000", 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4564e-05 2020-11-16T22:34:39.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059670+0000", 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013108299999999999 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059801+0000", 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66586 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560486715000003, 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146, 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:39.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5766999999999997e-05 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962333+0000", 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4960000000000001e-06 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962340+0000", 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.313e-05 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962353+0000", 2020-11-16T22:34:39.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.763e-06 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962358+0000", 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080385160000000008 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970396+0000", 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.08724875 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057645+0000", 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018116900000000001 2020-11-16T22:34:39.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057826+0000", 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063036259999999997 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064130+0000", 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030772299999999998 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064438+0000", 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5346000000000003e-05 2020-11-16T22:34:39.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064483+0000", 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8860000000000002e-05 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064512+0000", 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.688 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66587 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560371703000001, 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938, 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:39.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6044000000000002e-05 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962449+0000", 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9759999999999998e-06 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962456+0000", 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2969999999999999e-05 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962469+0000", 2020-11-16T22:34:39.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.977e-06 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962474+0000", 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079504459999999999 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970424+0000", 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.087276322000000003 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057700+0000", 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017350800000000001 2020-11-16T22:34:39.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057874+0000", 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013362096 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071236+0000", 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5955999999999998e-05 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071292+0000", 2020-11-16T22:34:39.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5148000000000005e-05 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071377+0000", 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2495000000000003e-05 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071450+0000", 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.698 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66589 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560125929000002, 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247, 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.699 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:39.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5683000000000003e-05 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962694+0000", 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2570000000000004e-06 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962700+0000", 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1571e-05 2020-11-16T22:34:39.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962712+0000", 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.454e-06 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962716+0000", 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100496154 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063212+0000", 2020-11-16T22:34:39.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111053 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063324+0000", 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020392000000000001 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063527+0000", 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007972771 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071500+0000", 2020-11-16T22:34:39.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9499999999999996e-06 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071510+0000", 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8403000000000001e-05 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071589+0000", 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1031e-05 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071640+0000", 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247 2020-11-16T22:34:39.910 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66590 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:39.911 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.560015655999997, 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329, 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:39.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1128999999999999e-05 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962800+0000", 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3040000000000001e-06 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962806+0000", 2020-11-16T22:34:39.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1032e-05 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962817+0000", 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962821+0000", 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104765866 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:39.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067587+0000", 2020-11-16T22:34:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106117 2020-11-16T22:34:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067693+0000", 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018714099999999999 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067880+0000", 2020-11-16T22:34:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049221739999999996 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.072802+0000", 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043900800000000001 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073241+0000", 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6645999999999999e-05 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073308+0000", 2020-11-16T22:34:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.426e-05 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073352+0000", 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66591 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.559911528999997, 2020-11-16T22:34:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996, 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962907+0000", 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0789999999999997e-06 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962913+0000", 2020-11-16T22:34:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0987e-05 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962924+0000", 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7900000000000001e-06 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962928+0000", 2020-11-16T22:34:40.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062214039999999998 2020-11-16T22:34:40.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969150+0000", 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010196699999999999 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969252+0000", 2020-11-16T22:34:40.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164537 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969416+0000", 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.089005452999999998 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058422+0000", 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003871518 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062293+0000", 2020-11-16T22:34:40.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3857000000000003e-05 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062347+0000", 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014316599999999999 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:40.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062490+0000", 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:40.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:40.336 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:40.336 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:40.338 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:40.351+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:40.338 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:40.437 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:40.452+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "pg_update_log_missing(4.1 epoch 199/171 rep_tid 56004 entries 199'11118 (0'0) error 4:98a9bd97:::1000000188d.0000000a:head by client.4471.0:91590 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 107'1100 roll_forward_to 199'11116)", 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:40.493982+0000", 2020-11-16T22:34:40.481 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 0.0022194010000000002, 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0022425610000000001, 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.493982+0000", 2020-11-16T22:34:40.482 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.493982+0000", 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7757999999999999e-05 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.483 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.494029+0000", 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3733e-05 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:40.484 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.494043+0000", 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1073e-05 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.494064+0000", 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1679999999999993e-06 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.485 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.494073+0000", 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.1097999999999998e-05 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:40.486 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:40.494124+0000", 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00014283000000000001 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: ], 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 1 2020-11-16T22:34:40.487 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:40.488 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:40.491 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:40.506 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:40.507 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:40.507 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:40.507 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:40.507 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:40.509 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:40.710 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:40.711 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:40.726+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:40.812 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:40.827+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:40.861 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:40.861 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:40.873 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:40.913 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:40.928+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:41.014 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:41.028+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:41.022 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.022 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.032 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:41.114 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.129+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.160 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:34:41.185 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.185 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.195 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:41.215 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.229+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.315 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.330+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.338 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.338 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.352 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:41.416 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.431+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.500 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.500 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.513 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:41.517 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:41.531+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:41.617 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.632+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.663 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.663 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.674 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:41.756 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.771+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.819 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.834+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:41.856 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:41.856 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:41.866 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:41.950 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:41.965+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.021 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.036+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~204s 2020-11-16T22:34:42.058 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:42.058 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:42.069 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:42.122 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.136+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:34:42.214 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:42.215 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:42.223 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.237+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.229 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:42.324 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:42.338+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:42.380 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:42.380 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:42.391 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:42.432 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:42.447+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:42.525 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:42.539+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:42.585 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:42.585 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:42.596 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:42.636 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:42.651+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:42.711 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:42.726 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.741+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.756 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:42.756 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:42.767 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:42.850 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.865+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.860 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:42.881 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:42.881 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:42.882 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:42.882 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:42.882 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:42.884 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:42.884 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:42.928 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:42.943+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:42.952 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:42.952 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:42.952 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:42.952 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:42.952 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:42.962 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:43.045 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:43.060+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:43.130 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:43.145+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:43.147 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:43.148 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 342.54782552699999, 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:43.150 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.151 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:43.152 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.153 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:43.154 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:43.155 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.156 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:43.157 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:43.158 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 284.251050453, 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.159 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.160 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.161 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.162 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:43.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:43.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.167 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 283.95267339399999, 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.168 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:43.169 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.170 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:43.171 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.172 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:43.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.174 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:43.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:43.176 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:43.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 223.07572537999999, 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.179 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.180 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.181 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.182 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.183 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:43.184 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.185 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.186 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.415287825999997, 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.187 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:43.188 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.189 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:43.190 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:43.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.191 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.191 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.191 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:43.191 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:43.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:43.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.399 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.415195280000006, 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:43.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:43.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:43.606 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.606 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.606 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:43.607 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.608 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:43.609 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.610 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:43.611 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:43.612 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.414790179999997, 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:43.613 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.614 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:43.615 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.616 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.818 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:43.818 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:43.818 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:43.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:43.819 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:43.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:43.821 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:43.822 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:43.823 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.414706215999999, 2020-11-16T22:34:44.026 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:44.026 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.026 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.027 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:44.028 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.029 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:44.030 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.031 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:44.032 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.033 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.412296338000004, 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.034 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.035 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.036 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.036 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:44.238 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:44.238 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.238 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.238 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:44.239 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.240 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:44.241 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:44.244 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.409570036999995, 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:44.245 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.246 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:44.247 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.248 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:44.249 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.250 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:44.251 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:44.252 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 80.407896214999994, 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:44.253 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.254 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:44.255 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.256 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:44.257 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.258 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:44.259 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:44.260 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:44.261 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.262 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 79.972148740999998, 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.263 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.264 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:44.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.266 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:44.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:44.268 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:44.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.373417080000003, 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.271 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.272 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:44.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.274 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:44.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.276 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.277 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:44.278 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.279 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.280 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.373343620999997, 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.281 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.282 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:44.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:44.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.287 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:44.288 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.289 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.290 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.372737796000003, 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.291 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:44.292 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.293 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:44.294 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.295 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:44.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.297 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:44.298 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 61.372580286999998, 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.299 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:44.300 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.301 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:44.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.303 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.304 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:44.305 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:44.306 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 37.199029867, 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.307 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:44.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.309 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:44.310 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.311 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:44.312 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:44.313 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.314 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.315 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:69916 3.2 3:545c15c7:::200.00000017:head [write 2999902~1932 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e195)", 2020-11-16T22:34:44.316 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 23.542457914, 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007, 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.317 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7635999999999997e-05 2020-11-16T22:34:44.318 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618576+0000", 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.319 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618583+0000", 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0594000000000001e-05 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618593+0000", 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2549999999999999e-06 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.320 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618598+0000", 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6642999999999999e-05 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.321 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618624+0000", 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9358000000000003e-05 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618684+0000", 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4988000000000006e-05 2020-11-16T22:34:44.322 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618779+0000", 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035862419999999999 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:44.323 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.622365+0000", 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.061623445999999998 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.683988+0000", 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7128e-05 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.324 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684035+0000", 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8116e-05 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684064+0000", 2020-11-16T22:34:44.325 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87626 4.6 e195/184 4:6090c0e3:::1000000188d.00000003:head v 195'3820, mlcod=195'3818)", 2020-11-16T22:34:44.326 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.528670071000001, 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874, 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000127615 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632454+0000", 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010485899999999999 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632559+0000", 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9750000000000007e-06 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632567+0000", 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7689999999999998e-06 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632571+0000", 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0516000000000003e-05 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632611+0000", 2020-11-16T22:34:44.331 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1466000000000001e-05 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632633+0000", 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.076359815999999997 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.332 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.708992+0000", 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2724000000000004e-05 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.709065+0000", 2020-11-16T22:34:44.333 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87627 4.4 e195/170 4:218b21fa:::1000000188d.00000001:head v 195'7868, mlcod=195'7866)", 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:44.334 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 22.528383059999999, 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005, 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:44.335 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9462000000000006e-05 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.336 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632693+0000", 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0729999999999999e-06 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632700+0000", 2020-11-16T22:34:44.337 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0679999999999999e-05 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632711+0000", 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3640000000000001e-06 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.338 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632715+0000", 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9465000000000002e-05 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632764+0000", 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9893000000000001e-05 2020-11-16T22:34:44.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632784+0000", 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078505851000000001 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.340 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711290+0000", 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3350000000000003e-05 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711344+0000", 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005 2020-11-16T22:34:44.341 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.342 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.342 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:44.342 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:44.342 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:44.343 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:34:44.344 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:44.345 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:44.358+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:44.444 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:44.459+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:44.511 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:92401 4.4 4:37415b28:::1000000188d.00000004:head [write 1024000~21992 [2731@0] in=21992b] snapc 1=[] ondisk+write+known_if_redirected e200)", 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.504354+0000", 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021613128999999998, 2020-11-16T22:34:44.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021639285000000001, 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 92401 2020-11-16T22:34:44.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504354+0000", 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504354+0000", 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7046999999999998e-05 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504411+0000", 2020-11-16T22:34:44.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2190000000000001e-06 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504417+0000", 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0848e-05 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504428+0000", 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5600000000000004e-06 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504432+0000", 2020-11-16T22:34:44.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088410260000000001 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.513273+0000", 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114789 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.513388+0000", 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018869700000000001 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.513577+0000", 2020-11-16T22:34:44.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091292470000000001 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522706+0000", 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018352433000000001 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76005 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.504528+0000", 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021438456000000002, 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021559223999999998, 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504528+0000", 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504528+0000", 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4286999999999998e-05 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504573+0000", 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5229999999999998e-06 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504580+0000", 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3329999999999999e-05 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504593+0000", 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6879999999999998e-06 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504598+0000", 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005137974 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.509736+0000", 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6532000000000004e-05 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.509823+0000", 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014081200000000001 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.509963+0000", 2020-11-16T22:34:44.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054351460000000001 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76007 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.528 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.504763+0000", 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021203403999999999, 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021395147, 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504763+0000", 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504763+0000", 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4715000000000001e-05 2020-11-16T22:34:44.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504808+0000", 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1489999999999997e-06 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504815+0000", 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2095e-05 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504827+0000", 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0720000000000001e-06 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504831+0000", 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013202538999999999 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.518034+0000", 2020-11-16T22:34:44.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9704999999999995e-05 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.518134+0000", 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014561000000000001 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.518279+0000", 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013515885 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76009 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.504990+0000", 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020976773000000001, 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021238018000000001, 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504990+0000", 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504990+0000", 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5016999999999999e-05 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505035+0000", 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5100000000000004e-06 2020-11-16T22:34:44.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505041+0000", 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6470999999999999e-05 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505058+0000", 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2980000000000002e-06 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505062+0000", 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010324760000000001 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.515387+0000", 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3445999999999999e-05 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.515480+0000", 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017049599999999999 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.515651+0000", 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010660998 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.542 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76011 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.505241+0000", 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020725754999999998, 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021056920999999999, 2020-11-16T22:34:44.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505241+0000", 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505241+0000", 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2540999999999997e-05 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505294+0000", 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0800000000000006e-06 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505302+0000", 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4732000000000001e-05 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505316+0000", 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5560000000000001e-06 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505321+0000", 2020-11-16T22:34:44.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.014741963 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.520063+0000", 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6122000000000002e-05 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.520129+0000", 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4146000000000003e-05 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.520223+0000", 2020-11-16T22:34:44.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01498214 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76012 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.505370+0000", 2020-11-16T22:34:44.550 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020597061, 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020996593000000001, 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505370+0000", 2020-11-16T22:34:44.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505370+0000", 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7206999999999997e-05 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505417+0000", 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7060000000000004e-06 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505425+0000", 2020-11-16T22:34:44.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4253999999999999e-05 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505439+0000", 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9149999999999998e-06 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505443+0000", 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018721258000000001 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.524164+0000", 2020-11-16T22:34:44.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6241999999999994e-05 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.524230+0000", 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108942 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.524339+0000", 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018969524000000001 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76013 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.505490+0000", 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020476645000000002, 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020944415000000001, 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505490+0000", 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505490+0000", 2020-11-16T22:34:44.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8003000000000001e-05 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505538+0000", 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9419999999999995e-06 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505546+0000", 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4483e-05 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505561+0000", 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.946e-06 2020-11-16T22:34:44.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505564+0000", 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017166280999999999 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522731+0000", 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118548 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522849+0000", 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017324699999999999 2020-11-16T22:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.523023+0000", 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.017532450000000002 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76015 4.5 4.5577df0d (undecoded) ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.505610+0000", 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020356896999999999, 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020893284000000002, 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505610+0000", 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505610+0000", 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7509000000000002e-05 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505657+0000", 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6259999999999997e-06 2020-11-16T22:34:44.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505664+0000", 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3213999999999999e-05 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505677+0000", 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5840000000000002e-06 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.505683+0000", 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2933000000000001e-05 2020-11-16T22:34:44.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(mds.0.3:76002 4.5 e200/117)", 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.521331+0000", 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0046356139999999997, 2020-11-16T22:34:44.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052200440000000001, 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521331+0000", 2020-11-16T22:34:44.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521331+0000", 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6041999999999997e-05 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521377+0000", 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7130000000000005e-06 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521385+0000", 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3192e-05 2020-11-16T22:34:44.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521398+0000", 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9040000000000003e-06 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.521405+0000", 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3850999999999995e-05 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(client.4471.0:92401 4.4 e200/170)", 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.522868+0000", 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0030991320000000001, 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003730798, 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522868+0000", 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522868+0000", 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1159000000000003e-05 2020-11-16T22:34:44.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522919+0000", 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3610000000000003e-06 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522926+0000", 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2815e-05 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522939+0000", 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.721e-06 2020-11-16T22:34:44.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.522948+0000", 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0055999999999994e-05 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:76002 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2731@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e200)", 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:44.504030+0000", 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021937070999999999, 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022620118000000002, 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504030+0000", 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504030+0000", 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8146999999999998e-05 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504078+0000", 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4300000000000003e-06 2020-11-16T22:34:44.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504084+0000", 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1756999999999999e-05 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504096+0000", 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5970000000000002e-06 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504101+0000", 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7005000000000002e-05 2020-11-16T22:34:44.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504138+0000", 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0341999999999997e-05 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504228+0000", 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043962599999999998 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:44.504668+0000", 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063790400000000001 2020-11-16T22:34:44.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 11 2020-11-16T22:34:44.588 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:44.590 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:44.590 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:44.605+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:44.690 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:44.705+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:44.786 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:44.787 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2020-11-16T22:34:44.787 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2020-11-16T22:34:44.787 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2020-11-16T22:34:44.787 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:44.791 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:44.806+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:44.796 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:44.884 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:44.892 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:44.907+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:44.952 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17250 3.0 3.ebe89178 (undecoded) ondisk+retry+write+known_if_redirected+full_force e108)", 2020-11-16T22:34:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 286.96512682299999, 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999, 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000068+0000", 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5399999999999999e-05 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000113+0000", 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2130000000000002e-06 2020-11-16T22:34:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000120+0000", 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3108000000000001e-05 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000134+0000", 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5990000000000002e-06 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000142+0000", 2020-11-16T22:34:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0325999999999999e-05 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000193+0000", 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5249999999999994e-06 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.000202+0000", 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024113658999999999 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024316+0000", 2020-11-16T22:34:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2074e-05 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.024328+0000", 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.88934911500000002 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913677+0000", 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6099999999999998e-05 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:58.913693+0000", 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91362511899999999 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:17251 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 285.75697665600001, 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005, 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208218+0000", 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010390700000000001 2020-11-16T22:34:44.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208322+0000", 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.97e-05 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208352+0000", 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7002000000000002e-05 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208389+0000", 2020-11-16T22:34:44.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7159999999999998e-05 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208406+0000", 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000166446 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208572+0000", 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0736e-05 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.208583+0000", 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.76361938299999998 2020-11-16T22:34:44.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.972202+0000", 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001328819 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for missing object", 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:29:59.973531+0000", 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042060443000000003 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015592+0000", 2020-11-16T22:34:44.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000130304 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for rw locks", 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.015722+0000", 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066128660000000002 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022335+0000", 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115659 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.022450+0000", 2020-11-16T22:34:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006618667 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.029069+0000", 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034487559999999999 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.032518+0000", 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002181451 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:44.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034699+0000", 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2639999999999995e-05 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034782+0000", 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141755 2020-11-16T22:34:44.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:30:00.034924+0000", 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.82670569400000005 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:24037 3.5 3:b02ffd62:::200.00000010:head [write 477553~1926 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e140)", 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 222.421359061, 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001, 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543836+0000", 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9966000000000003e-05 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543876+0000", 2020-11-16T22:34:44.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1559999999999995e-06 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543884+0000", 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0824e-05 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543895+0000", 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5509999999999999e-06 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543900+0000", 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6257999999999997e-05 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.543936+0000", 2020-11-16T22:34:44.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5112999999999998e-05 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544001+0000", 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9536000000000004e-05 2020-11-16T22:34:44.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.544101+0000", 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001650924 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:02.545752+0000", 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.55522068999999996 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100973+0000", 2020-11-16T22:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1580000000000005e-06 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:03.100981+0000", 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.013923516 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.114904+0000", 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118777 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:04.115023+0000", 2020-11-16T22:34:44.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5711874690000001 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:47852 4.0 4:0caefcac:::1000000188a.00000001:head [write 2347008~24576 [1881@2479577]] snapc 1=[] ondisk+write+known_if_redirected e152)", 2020-11-16T22:34:44.989 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 193.94054840800001, 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022, 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:44.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 47852 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:44.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024646+0000", 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7007000000000004e-05 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024713+0000", 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024721+0000", 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3522e-05 2020-11-16T22:34:44.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024734+0000", 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1460000000000001e-06 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:44.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.024740+0000", 2020-11-16T22:34:44.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000646359 2020-11-16T22:34:44.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025387+0000", 2020-11-16T22:34:44.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0516999999999999e-05 2020-11-16T22:34:44.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:31.025397+0000", 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0128770309999999 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:44.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038274+0000", 2020-11-16T22:34:44.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000141471 2020-11-16T22:34:44.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:44.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:44.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038416+0000", 2020-11-16T22:34:44.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121727 2020-11-16T22:34:44.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:44.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.038538+0000", 2020-11-16T22:34:45.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003939723 2020-11-16T22:34:45.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042477+0000", 2020-11-16T22:34:45.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037657499999999998 2020-11-16T22:34:45.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042854+0000", 2020-11-16T22:34:45.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1348999999999998e-05 2020-11-16T22:34:45.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042915+0000", 2020-11-16T22:34:45.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3106999999999998e-05 2020-11-16T22:34:45.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:31:32.042968+0000", 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.018322022 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:41604 3.4 3:3939b497:::200.00000014:head [write 808192~3309 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e171)", 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 126.001116306, 2020-11-16T22:34:45.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996, 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:45.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964078+0000", 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3495999999999998e-05 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964122+0000", 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6729999999999998e-06 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964131+0000", 2020-11-16T22:34:45.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4209999999999999e-05 2020-11-16T22:34:45.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964145+0000", 2020-11-16T22:34:45.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2020-11-16T22:34:45.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964152+0000", 2020-11-16T22:34:45.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6425999999999999e-05 2020-11-16T22:34:45.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964199+0000", 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1729000000000001e-05 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:38.964210+0000", 2020-11-16T22:34:45.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99162645900000002 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955837+0000", 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013944599999999999 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.955976+0000", 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000201917 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.956178+0000", 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026961559999999999 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.958874+0000", 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030118279999999998 2020-11-16T22:34:45.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961886+0000", 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6498e-05 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.961943+0000", 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4783000000000005e-05 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:32:39.962028+0000", 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99794913799999996 2020-11-16T22:34:45.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:74555 4.0 4:1125f8c3:::1000000188d.00000006:head [read 2052096~0 [1326@4194304]] snapc 0=[] ondisk+read+known_if_redirected e183)", 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:45.019 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.776201983999997, 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997, 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2020-11-16T22:34:45.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 74555 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.188993+0000", 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2381000000000001e-05 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189015+0000", 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189020+0000", 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7239999999999998e-06 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189029+0000", 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.938e-06 2020-11-16T22:34:45.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189034+0000", 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2181999999999997e-05 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189076+0000", 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5170000000000005e-06 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189085+0000", 2020-11-16T22:34:45.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77183806200000005 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.960923+0000", 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108077 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961031+0000", 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012471699999999999 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961156+0000", 2020-11-16T22:34:45.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77216345799999997 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:54750 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 1326@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:45.029 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.776053601000001, 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005, 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189141+0000", 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4946000000000001e-05 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189166+0000", 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0829999999999998e-06 2020-11-16T22:34:45.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189173+0000", 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4460000000000004e-06 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189181+0000", 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5739999999999998e-06 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189183+0000", 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1687999999999999e-05 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189215+0000", 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5220000000000007e-06 2020-11-16T22:34:45.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.189223+0000", 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77220181499999996 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961425+0000", 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109646 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961535+0000", 2020-11-16T22:34:45.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124568 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.961659+0000", 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054203020000000001 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967080+0000", 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089180100000000001 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.967971+0000", 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5658e-05 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968047+0000", 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1239999999999997e-05 2020-11-16T22:34:45.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:33:23.968098+0000", 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77895728900000005 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66577 4.0 4:1125f8c3:::1000000188d.00000006:head [trimtrunc 2104@3880336] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003771915000002, 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634, 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.042 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961423+0000", 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6947000000000001e-05 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961450+0000", 2020-11-16T22:34:45.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7630000000000003e-06 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961456+0000", 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2191999999999999e-05 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961469+0000", 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5969999999999999e-06 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961474+0000", 2020-11-16T22:34:45.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5272999999999999e-05 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961510+0000", 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100571 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961610+0000", 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176787 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961787+0000", 2020-11-16T22:34:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085900679999999993 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970377+0000", 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.091642535999999997 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062020+0000", 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0728000000000004e-05 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062100+0000", 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2172000000000001e-05 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062152+0000", 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100729634 2020-11-16T22:34:45.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66578 4.4 4:3be3f0ca:::1000000188d.00000007:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:45.052 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003692242, 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003, 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:45.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961502+0000", 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3958e-05 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961526+0000", 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.014e-06 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961530+0000", 2020-11-16T22:34:45.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7959999999999998e-06 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961537+0000", 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.897e-06 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961540+0000", 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5605000000000002e-05 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961576+0000", 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5517000000000003e-05 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961641+0000", 2020-11-16T22:34:45.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010893899999999999 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961750+0000", 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0073697459999999999 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969120+0000", 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088765291999999996 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057885+0000", 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6371000000000004e-05 2020-11-16T22:34:45.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057972+0000", 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013677599999999999 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058108+0000", 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096605911000000003 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66579 4.0 4:19ad6e7d:::1000000188d.00000008:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003625827999997, 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999, 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961569+0000", 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6403000000000001e-05 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961595+0000", 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9859999999999996e-06 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961599+0000", 2020-11-16T22:34:45.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2020-11-16T22:34:45.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961606+0000", 2020-11-16T22:34:45.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4480000000000001e-06 2020-11-16T22:34:45.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961609+0000", 2020-11-16T22:34:45.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049254529999999998 2020-11-16T22:34:45.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966534+0000", 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7503999999999997e-05 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966592+0000", 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7417999999999995e-05 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966689+0000", 2020-11-16T22:34:45.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095498420000000001 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062187+0000", 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4015000000000003e-05 2020-11-16T22:34:45.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062251+0000", 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1647e-05 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062323+0000", 2020-11-16T22:34:45.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2119e-05 2020-11-16T22:34:45.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062375+0000", 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10080626199999999 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66580 4.5 4:ab64ebfc:::1000000188d.00000009:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003558253999998, 2020-11-16T22:34:45.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688, 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961636+0000", 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4732999999999999e-05 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961661+0000", 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0949999999999998e-06 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961665+0000", 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9530000000000002e-06 2020-11-16T22:34:45.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961672+0000", 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7920000000000004e-06 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961682+0000", 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0902999999999998e-05 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961723+0000", 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8277999999999999e-05 2020-11-16T22:34:45.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961791+0000", 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010717000000000001 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961898+0000", 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10533300399999999 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067231+0000", 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1401e-05 2020-11-16T22:34:45.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067243+0000", 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1019999999999993e-05 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067324+0000", 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5339e-05 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067379+0000", 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105742688 2020-11-16T22:34:45.086 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66581 4.1 4:98a9bd97:::1000000188d.0000000a:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003475588000001, 2020-11-16T22:34:45.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006, 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:45.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961719+0000", 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7623999999999998e-05 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961747+0000", 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961751+0000", 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0149999999999998e-06 2020-11-16T22:34:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961758+0000", 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2649999999999998e-06 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961762+0000", 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4431000000000002e-05 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961807+0000", 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035119090000000001 2020-11-16T22:34:45.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965319+0000", 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012228 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965441+0000", 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092836962999999995 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058278+0000", 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013207889999999999 2020-11-16T22:34:45.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059599+0000", 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2479999999999996e-05 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059681+0000", 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5322999999999995e-05 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059776+0000", 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098057395000000006 2020-11-16T22:34:45.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66582 4.1 4:9ff04a26:::1000000188d.0000000b:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003379805000002, 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302, 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:45.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961815+0000", 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0484e-05 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961865+0000", 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1609999999999997e-06 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961873+0000", 2020-11-16T22:34:45.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2751999999999999e-05 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961885+0000", 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4719999999999997e-06 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961890+0000", 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080703010000000002 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969960+0000", 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7603999999999999e-05 2020-11-16T22:34:45.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970018+0000", 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010071199999999999 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970118+0000", 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.088442494999999996 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058561+0000", 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037181409999999999 2020-11-16T22:34:45.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062279+0000", 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3547000000000001e-05 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062333+0000", 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3633000000000004e-05 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062416+0000", 2020-11-16T22:34:45.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100601302 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66583 4.5 4:a7565345:::1000000188d.0000000c:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:45.107 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003257237, 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072, 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:45.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961937+0000", 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6323999999999999e-05 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961984+0000", 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.109e-06 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.961991+0000", 2020-11-16T22:34:45.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2452999999999999e-05 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962003+0000", 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4100000000000001e-06 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962008+0000", 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046175440000000003 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966625+0000", 2020-11-16T22:34:45.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0504999999999998e-05 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966686+0000", 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3917000000000002e-05 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.966780+0000", 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100608974 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067389+0000", 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2806999999999999e-05 2020-11-16T22:34:45.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067431+0000", 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6773999999999994e-05 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067508+0000", 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1255000000000003e-05 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067560+0000", 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105622072 2020-11-16T22:34:45.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66585 4.4 4:39e71ca2:::1000000188d.0000000e:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.003026677999998, 2020-11-16T22:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994, 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:45.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962168+0000", 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5401000000000001e-05 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962213+0000", 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1690000000000003e-06 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962221+0000", 2020-11-16T22:34:45.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2514000000000001e-05 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962233+0000", 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.378e-06 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962238+0000", 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032176309999999999 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965456+0000", 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000101195 2020-11-16T22:34:45.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965557+0000", 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149464 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.965707+0000", 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092509202999999998 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058216+0000", 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013792990000000001 2020-11-16T22:34:45.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059595+0000", 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4564e-05 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059670+0000", 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013108299999999999 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.059801+0000", 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097632900999999994 2020-11-16T22:34:45.126 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66586 4.0 4:00fd5ff3:::1000000188d.0000000f:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.002907123999996, 2020-11-16T22:34:45.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146, 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:45.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962288+0000", 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5766999999999997e-05 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962333+0000", 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4960000000000001e-06 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962340+0000", 2020-11-16T22:34:45.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.313e-05 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962353+0000", 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.763e-06 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962358+0000", 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080385160000000008 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970396+0000", 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.08724875 2020-11-16T22:34:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057645+0000", 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018116900000000001 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057826+0000", 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063036259999999997 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064130+0000", 2020-11-16T22:34:45.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00030772299999999998 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064438+0000", 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5346000000000003e-05 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064483+0000", 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8860000000000002e-05 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.064512+0000", 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102224146 2020-11-16T22:34:45.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66587 4.5 4:ab344873:::1000000188d.00000010:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:45.137 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.002792112000002, 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938, 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:45.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962403+0000", 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6044000000000002e-05 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962449+0000", 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9759999999999998e-06 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962456+0000", 2020-11-16T22:34:45.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2969999999999999e-05 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962469+0000", 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.977e-06 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962474+0000", 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079504459999999999 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.970424+0000", 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.087276322000000003 2020-11-16T22:34:45.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057700+0000", 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017350800000000001 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.057874+0000", 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013362096 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071236+0000", 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5955999999999998e-05 2020-11-16T22:34:45.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071292+0000", 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5148000000000005e-05 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071377+0000", 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2495000000000003e-05 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071450+0000", 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.109046938 2020-11-16T22:34:45.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66589 4.5 4:ae4c5600:::1000000188d.00000012:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.002546338000002, 2020-11-16T22:34:45.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247, 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:45.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962648+0000", 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5683000000000003e-05 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962694+0000", 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2570000000000004e-06 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962700+0000", 2020-11-16T22:34:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1571e-05 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962712+0000", 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.454e-06 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962716+0000", 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100496154 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063212+0000", 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111053 2020-11-16T22:34:45.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063324+0000", 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020392000000000001 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.063527+0000", 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007972771 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071500+0000", 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9499999999999996e-06 2020-11-16T22:34:45.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071510+0000", 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8403000000000001e-05 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071589+0000", 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1031e-05 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.071640+0000", 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.108991247 2020-11-16T22:34:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66590 4.5 4:a81cf227:::1000000188d.00000013:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.002436064999998, 2020-11-16T22:34:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329, 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:45.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962759+0000", 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1128999999999999e-05 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962800+0000", 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3040000000000001e-06 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962806+0000", 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1032e-05 2020-11-16T22:34:45.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962817+0000", 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962821+0000", 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104765866 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067587+0000", 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106117 2020-11-16T22:34:45.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067693+0000", 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018714099999999999 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.067880+0000", 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049221739999999996 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.072802+0000", 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043900800000000001 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073241+0000", 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6645999999999999e-05 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073308+0000", 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.426e-05 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.073352+0000", 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.11059329 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.3:66591 4.4 4:38fa980f:::1000000188d.00000014:head [trimtrunc 2104@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.002331937999998, 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996, 2020-11-16T22:34:45.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:45.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962863+0000", 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4579000000000001e-05 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:45.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962907+0000", 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0789999999999997e-06 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962913+0000", 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0987e-05 2020-11-16T22:34:45.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962924+0000", 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7900000000000001e-06 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:45.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.962928+0000", 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062214039999999998 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969150+0000", 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010196699999999999 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969252+0000", 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000164537 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:05.969416+0000", 2020-11-16T22:34:45.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.089005452999999998 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.058422+0000", 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003871518 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062293+0000", 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3857000000000003e-05 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062347+0000", 2020-11-16T22:34:45.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014316599999999999 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:06.062490+0000", 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099627336999999996 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:45.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:45.177 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:45.179 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:45.179 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:45.179 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:45.180 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:45.180 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:45.180 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:45.180 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:45.180 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:45.181 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:34:45.181 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:45.193+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:45.181 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:45.278 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:45.293+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:45.328 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:45.328 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:45.339 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:45.379 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.394+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.480 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.494+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.520 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:45.520 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:45.530 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:45.581 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.595+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.678 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:45.679 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:45.682 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.696+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.689 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:45.783 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.797+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.835 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:45.835 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:45.847 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:45.888 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.903+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:45.985 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:45.999+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:46.035 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.035 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.046 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:46.087 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:46.101+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:46.187 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:46.201+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:46.244 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.244 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.254 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:46.297 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:46.312+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:46.389 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:46.403+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:46.398 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.398 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.409 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:46.493 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:46.508+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:46.588 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.588 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.590 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:46.604+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:46.599 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:46.691 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:46.705+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:46.784 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.784 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.792 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:46.807+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:46.796 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:46.893 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:46.908+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:46.960 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:46.961 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:46.971 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:46.993 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:47.008+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:47.095 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:47.109+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:47.116 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:47.116 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:47.130 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:47.181 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T22:34:47.181 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T22:34:47.213 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:47.228+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:47.296 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:47.311+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:47.323 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:47.324 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:47.331 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:47.336 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:47.352 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:47.352 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:47.352 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:47.352 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T22:34:47.352 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T22:34:47.354 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:47.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:47.397 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:47.412+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:47.489 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:47.489 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:47.489 INFO:teuthology.orchestra.run.smithi194.stdout: "num_ops": 0 2020-11-16T22:34:47.489 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:47.498 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:47.513+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:47.501 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:47.599 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:47.614+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:47.682 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:47.682 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [], 2020-11-16T22:34:47.683 INFO:teuthology.orchestra.run.smithi194.stdout: "complaint_time": 30, 2020-11-16T22:34:47.683 INFO:teuthology.orchestra.run.smithi194.stdout: "num_blocked_ops": 0 2020-11-16T22:34:47.683 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:47.694 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:47.737 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:47.752+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:47.800 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:47.815+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:47.840 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~209s 2020-11-16T22:34:47.847 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:34:47.847 INFO:teuthology.orchestra.run.smithi194.stdout: "size": 20, 2020-11-16T22:34:47.847 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 600, 2020-11-16T22:34:47.847 INFO:teuthology.orchestra.run.smithi194.stdout: "ops": [ 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:7921 3.3 3:ceffbd45:::200.0000000a:head [write 2438656~1925 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e96)", 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 347.24761837300002, 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999, 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:47.848 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.849 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613171+0000", 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.6721999999999998e-05 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613208+0000", 2020-11-16T22:34:47.850 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.0379999999999996e-06 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613214+0000", 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.9209999999999998e-06 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.851 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.613223+0000", 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052594239000000001 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665817+0000", 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.596e-05 2020-11-16T22:34:47.852 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665863+0000", 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5494000000000001e-05 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.853 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.665918+0000", 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.7368999999999996e-05 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 0", 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.666006+0000", 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0015170209999999999 2020-11-16T22:34:47.854 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667523+0000", 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3680000000000001e-06 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.855 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667528+0000", 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0022000000000003e-05 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667578+0000", 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012390599999999999 2020-11-16T22:34:47.856 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:00.667702+0000", 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.054531059999999999 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:47.857 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17250 3.0 3:1e8917d7:::200.0000000e:head [write 86227~1931 [fadvise_dontneed]] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.95084329899998, 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798, 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:47.858 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.859 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909946+0000", 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8114999999999997e-05 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909984+0000", 2020-11-16T22:34:47.860 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8830000000000003e-06 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.909990+0000", 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.3198e-05 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.861 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910003+0000", 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.4800000000000001e-06 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910012+0000", 2020-11-16T22:34:47.862 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6780999999999997e-05 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910068+0000", 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.1910000000000006e-06 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.863 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op must wait for map", 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.910078+0000", 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0045798749999999997 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914658+0000", 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4599e-05 2020-11-16T22:34:47.864 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:58.914672+0000", 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1892645879999999 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.865 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.103937+0000", 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000148088 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104085+0000", 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.194138798 2020-11-16T22:34:47.866 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:17252 3.0 3:1e8917d7:::200.0000000e:head [write 88158~1105 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e109)", 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 288.65246624000002, 2020-11-16T22:34:47.867 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003, 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:47.868 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:47.869 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:47.869 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:47.869 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:47.869 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208323+0000", 2020-11-16T22:34:48.070 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000117285 2020-11-16T22:34:48.070 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.070 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.070 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:48.070 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208440+0000", 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.161e-06 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208447+0000", 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.2908000000000001e-05 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208459+0000", 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.7540000000000006e-06 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:48.072 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208467+0000", 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.2000000000000001e-05 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208549+0000", 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1063000000000001e-05 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.073 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:29:59.208560+0000", 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.89566678799999999 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:48.074 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104227+0000", 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00018677599999999999 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104414+0000", 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00022982300000000001 2020-11-16T22:34:48.075 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.104644+0000", 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043141209999999998 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:48.076 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.108958+0000", 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002132466 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111090+0000", 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1456000000000002e-05 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.077 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111152+0000", 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00012002099999999999 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:48.078 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:30:00.111272+0000", 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.90294862200000003 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.079 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:23586 3.0 e139/111 3:17051619:::200.0000000f:head v 139'8598, mlcod=139'8596)", 2020-11-16T22:34:48.282 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:48.282 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 227.775518226, 2020-11-16T22:34:48.282 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034, 2020-11-16T22:34:48.282 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.283 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085271+0000", 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4143000000000002e-05 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:48.284 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085305+0000", 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3210000000000004e-06 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085311+0000", 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.8609999999999998e-06 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.285 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085318+0000", 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.194e-06 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:48.286 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085323+0000", 2020-11-16T22:34:48.287 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8951e-05 2020-11-16T22:34:48.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.287 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.490 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:48.490 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085362+0000", 2020-11-16T22:34:48.490 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1087e-05 2020-11-16T22:34:48.490 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.490 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.085383+0000", 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.051847978000000003 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:48.491 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137231+0000", 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6499000000000003e-05 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:31:00.137277+0000", 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052006034 2020-11-16T22:34:48.492 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54637 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.115080672000005, 2020-11-16T22:34:48.493 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522, 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:48.494 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:48.495 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.495 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.698 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:48.698 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745709+0000", 2020-11-16T22:34:48.698 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3302000000000002e-05 2020-11-16T22:34:48.698 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.698 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745732+0000", 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2209999999999996e-06 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745737+0000", 2020-11-16T22:34:48.699 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5310000000000007e-06 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745747+0000", 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1489999999999996e-06 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.700 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745751+0000", 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.1612999999999994e-05 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745812+0000", 2020-11-16T22:34:48.701 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6754999999999994e-05 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745879+0000", 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.8280000000000001e-05 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.702 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745978+0000", 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053038453999999999 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799016+0000", 2020-11-16T22:34:48.703 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000114877 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799131+0000", 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3762000000000004e-05 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.704 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799195+0000", 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.4577999999999998e-05 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799249+0000", 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053540522 2020-11-16T22:34:48.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54639 4.3 4:cf4ec11c:::1000000188d.00000017:head [trimtrunc 1319@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e182)", 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:48.706 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.114988126, 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997, 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.707 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:48.910 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745801+0000", 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4324999999999998e-05 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745826+0000", 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9779999999999996e-06 2020-11-16T22:34:48.911 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745831+0000", 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0339999999999999e-06 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:48.912 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745838+0000", 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6419999999999999e-06 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.745840+0000", 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043644809999999999 2020-11-16T22:34:48.913 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750205+0000", 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2547999999999997e-05 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:48.914 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:49.118 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750277+0000", 2020-11-16T22:34:49.118 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010177 2020-11-16T22:34:49.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.118 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.750379+0000", 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.048913058000000002 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:49.119 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799292+0000", 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.002241505 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801534+0000", 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0511000000000003e-05 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.120 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801604+0000", 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6638999999999998e-05 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:49.121 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.801661+0000", 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055859490999999997 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54631 4.0 e182/161 4:00fd5ff3:::1000000188d.0000000f:head v 182'9937, mlcod=182'9935)", 2020-11-16T22:34:49.122 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:49.123 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.114583026000005, 2020-11-16T22:34:49.123 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998, 2020-11-16T22:34:49.326 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:49.326 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:49.326 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:49.326 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:49.327 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746206+0000", 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3089000000000001e-05 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746239+0000", 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1579999999999998e-06 2020-11-16T22:34:49.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746244+0000", 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.392e-06 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:49.329 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746250+0000", 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.402e-06 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746252+0000", 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.3275999999999998e-05 2020-11-16T22:34:49.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.331 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.331 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:49.331 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746286+0000", 2020-11-16T22:34:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6311e-05 2020-11-16T22:34:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746302+0000", 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.007543442 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753845+0000", 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045102995 2020-11-16T22:34:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798948+0000", 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.052742064999999998 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54632 4.5 e182/117 4:ab344873:::1000000188d.00000010:head v 182'10309, mlcod=182'10307)", 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.114499061999993, 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998, 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746290+0000", 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2375000000000001e-05 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746323+0000", 2020-11-16T22:34:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4700000000000004e-06 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746327+0000", 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.3890000000000002e-06 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746334+0000", 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.9380000000000001e-06 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746336+0000", 2020-11-16T22:34:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9759999999999999e-05 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746376+0000", 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3828e-05 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.746400+0000", 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053074631999999997 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799475+0000", 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9118e-05 2020-11-16T22:34:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.746 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.746 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:49.746 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799524+0000", 2020-11-16T22:34:49.746 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.053233509999999998 2020-11-16T22:34:49.746 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54636 4.4 e182/170 4:38fa980f:::1000000188d.00000014:head v 182'4866, mlcod=182'4863)", 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:49.747 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.112089183999998, 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001, 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:49.748 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748700+0000", 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6830999999999998e-05 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.749 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748757+0000", 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1269999999999997e-06 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748764+0000", 2020-11-16T22:34:49.750 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.487e-05 2020-11-16T22:34:49.751 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.954 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.954 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:49.954 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748779+0000", 2020-11-16T22:34:49.954 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.4499999999999997e-06 2020-11-16T22:34:49.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748783+0000", 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.4904999999999999e-05 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:49.955 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748818+0000", 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6313000000000001e-05 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.748835+0000", 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0050563589999999999 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.956 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753891+0000", 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.045089377999999999 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:49.957 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.798980+0000", 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050280233000000001 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:54634 4.5 e182/117 4:ae4c5600:::1000000188d.00000012:head v 182'10310, mlcod=182'10307)", 2020-11-16T22:34:49.958 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:50.162 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.109362883000003, 2020-11-16T22:34:50.162 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002, 2020-11-16T22:34:50.162 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.163 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751426+0000", 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.5324999999999998e-05 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.164 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751462+0000", 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.7100000000000004e-06 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751467+0000", 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9379999999999992e-06 2020-11-16T22:34:50.165 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751475+0000", 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1409999999999998e-06 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.166 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751480+0000", 2020-11-16T22:34:50.167 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1646000000000002e-05 2020-11-16T22:34:50.167 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.167 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.167 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751511+0000", 2020-11-16T22:34:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5268e-05 2020-11-16T22:34:50.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.751526+0000", 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055770581 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.371 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807297+0000", 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3253000000000001e-05 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.807340+0000", 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.055913862000000002 2020-11-16T22:34:50.372 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(client.4471.0:74462 4.6 4:6090c0e3:::1000000188d.00000003:head [write 770657~51593 [1318@4194304]] snapc 1=[] ondisk+write+known_if_redirected e182)", 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:50.373 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 85.107689061000002, 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998, 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "client_info": { 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "client": "client.4471", 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "client_addr": "192.168.0.1:0/2424814314", 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: "tid": 74462 2020-11-16T22:34:50.374 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.375 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753100+0000", 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.6539000000000002e-05 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753157+0000", 2020-11-16T22:34:50.376 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0749999999999997e-06 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753162+0000", 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.0860000000000008e-06 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.377 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753171+0000", 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.476e-06 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753175+0000", 2020-11-16T22:34:50.378 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.6159999999999999e-05 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753222+0000", 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010139100000000001 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.379 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753323+0000", 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000182275 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.753505+0000", 2020-11-16T22:34:50.380 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.046297086000000001 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.799802+0000", 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0039610169999999998 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.381 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803763+0000", 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9394000000000006e-05 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803843+0000", 2020-11-16T22:34:50.382 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000145167 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:22.803988+0000", 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.050887665999999998 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.383 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:54748 4.6 4.dc095ce6 (undecoded) ondisk+write+known_if_redirected+full_force e183)", 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 84.671941587000006, 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001, 2020-11-16T22:34:50.384 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "reached pg", 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188848+0000", 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5109e-05 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.386 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188873+0000", 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.5990000000000001e-06 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188878+0000", 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.1423000000000001e-05 2020-11-16T22:34:50.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188890+0000", 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9190000000000004e-06 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.388 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188896+0000", 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.5188e-05 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188961+0000", 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.4880000000000001e-06 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.389 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for peered", 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.188968+0000", 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76233165800000002 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.390 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951300+0000", 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.5028000000000001e-05 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:23.951315+0000", 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.76246741200000001 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.391 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59683 4.3 4:db4b432d:::1000000188d.00000005:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 66.073209926000004, 2020-11-16T22:34:50.392 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999, 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787579+0000", 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.5083999999999999e-05 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.394 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787604+0000", 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.2950000000000001e-06 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787610+0000", 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.5550000000000003e-06 2020-11-16T22:34:50.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787618+0000", 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.1810000000000003e-06 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.396 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787623+0000", 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9626999999999999e-05 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787662+0000", 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.8029999999999998e-05 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.397 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787720+0000", 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.5656999999999996e-05 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:50.398 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787816+0000", 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065850878000000002 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.853667+0000", 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00178105 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.399 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855448+0000", 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0654999999999994e-05 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.400 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855518+0000", 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0136999999999998e-05 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.855569+0000", 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067989148999999999 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.401 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:59699 4.3 4:cc94663d:::1000000188d.00000015:head [trimtrunc 1661@0] snapc 1=[] ondisk+write+known_if_redirected+full_force e188)", 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 66.073136466999998, 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994, 2020-11-16T22:34:50.402 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787653+0000", 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.3790000000000001e-05 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.404 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787677+0000", 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.8500000000000004e-06 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787681+0000", 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7469999999999999e-06 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.405 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787687+0000", 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.446e-06 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.406 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.787690+0000", 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.059994367 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847684+0000", 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000133497 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.407 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.847818+0000", 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00020062199999999999 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:50.408 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.848018+0000", 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.010134239 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858152+0000", 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00060789999999999998 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.409 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858760+0000", 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2383000000000001e-05 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.410 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858833+0000", 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9146e-05 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.858882+0000", 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.071228986999999994 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.411 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59679 4.4 e188/170 4:218b21fa:::1000000188d.00000001:head v 188'5815, mlcod=188'5813)", 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 66.072530642000004, 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003, 2020-11-16T22:34:50.412 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788259+0000", 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.1707e-05 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.414 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788290+0000", 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.0420000000000002e-06 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788295+0000", 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1339999999999998e-06 2020-11-16T22:34:50.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788303+0000", 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.444e-06 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.416 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788306+0000", 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9891999999999997e-05 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788346+0000", 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6084999999999998e-05 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.417 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788372+0000", 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067795032000000005 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.418 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856167+0000", 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.9675000000000002e-05 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.856207+0000", 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.067948011000000003 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.419 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(mds.0.3:59680 4.2 e188/163 4:5e663dd9:::1000000188d.00000002:head v 188'2147, mlcod=188'2145)", 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 66.072373132999999, 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004, 2020-11-16T22:34:50.420 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.421 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788416+0000", 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9299000000000003e-05 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.422 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788475+0000", 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.1760000000000004e-06 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788483+0000", 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0949e-05 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.423 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788494+0000", 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.442e-06 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.424 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788498+0000", 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3062000000000002e-05 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788541+0000", 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.6600999999999999e-05 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.425 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.788558+0000", 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062615768000000002 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851173+0000", 2020-11-16T22:34:50.426 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.7547999999999998e-05 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:33:41.851241+0000", 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.062824845000000004 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.427 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:66593 3.2 3:545c15c7:::200.00000017:head [write 1181944~1927 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e192)", 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 41.898822713000001, 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001, 2020-11-16T22:34:50.428 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.429 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.961967+0000", 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.5807000000000001e-05 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.430 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962042+0000", 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.4420999999999999e-05 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962067+0000", 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.4737e-05 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.431 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962082+0000", 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3229999999999997e-06 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962087+0000", 2020-11-16T22:34:50.432 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.3761000000000001e-05 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962161+0000", 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.2246e-05 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.433 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962223+0000", 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 8.6142000000000003e-05 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:50.434 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.962309+0000", 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0043872290000000003 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:05.966696+0000", 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.091135593000000001 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.435 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057832+0000", 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9246000000000003e-05 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057881+0000", 2020-11-16T22:34:50.436 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.2447999999999998e-05 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:06.057914+0000", 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.095946953000000001 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.437 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_op(mds.0.3:69916 3.2 3:545c15c7:::200.00000017:head [write 2999902~1932 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e195)", 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 28.242250760000001, 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007, 2020-11-16T22:34:50.438 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618539+0000", 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7635999999999997e-05 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.440 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618576+0000", 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 6.6340000000000003e-06 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618583+0000", 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0594000000000001e-05 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.441 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618593+0000", 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.2549999999999999e-06 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.442 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618598+0000", 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.6642999999999999e-05 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618624+0000", 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.9358000000000003e-05 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.443 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618684+0000", 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 9.4988000000000006e-05 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "waiting for subops from 2", 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.618779+0000", 2020-11-16T22:34:50.444 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.0035862419999999999 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "op_commit", 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.622365+0000", 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.061623445999999998 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.445 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "sub_op_commit_rec", 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.683988+0000", 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.7128e-05 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684035+0000", 2020-11-16T22:34:50.446 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.8116e-05 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:19.684064+0000", 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.065525040000000007 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87626 4.6 e195/184 4:6090c0e3:::1000000188d.00000003:head v 195'3820, mlcod=195'3818)", 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.228462917000002, 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874, 2020-11-16T22:34:50.448 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.449 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632326+0000", 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.000127615 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632454+0000", 2020-11-16T22:34:50.450 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.00010485899999999999 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632559+0000", 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9750000000000007e-06 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.451 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632567+0000", 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 3.7689999999999998e-06 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632571+0000", 2020-11-16T22:34:50.452 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.0516000000000003e-05 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632611+0000", 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 2.1466000000000001e-05 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.453 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632633+0000", 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.076359815999999997 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.708992+0000", 2020-11-16T22:34:50.454 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.2724000000000004e-05 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.709065+0000", 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.07673874 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.455 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "description": "osd_repop(client.4471.0:87627 4.4 e195/170 4:218b21fa:::1000000188d.00000001:head v 195'7868, mlcod=195'7866)", 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "initiated_at": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "age": 27.228175906000001, 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005, 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "type_data": { 2020-11-16T22:34:50.456 INFO:teuthology.orchestra.run.smithi194.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "events": [ 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "initiated", 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.457 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "throttled", 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632613+0000", 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.9462000000000006e-05 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "header_read", 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632693+0000", 2020-11-16T22:34:50.458 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 7.0729999999999999e-06 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "all_read", 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632700+0000", 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.0679999999999999e-05 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.459 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "dispatched", 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632711+0000", 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.3640000000000001e-06 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632715+0000", 2020-11-16T22:34:50.460 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 4.9465000000000002e-05 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "reached_pg", 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632764+0000", 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 1.9893000000000001e-05 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "started", 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.632784+0000", 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078505851000000001 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "commit_sent", 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711290+0000", 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 5.3350000000000003e-05 2020-11-16T22:34:50.462 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: { 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "event": "done", 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "time": "2020-11-16T22:34:20.711344+0000", 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: "duration": 0.078730138000000005 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.463 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: } 2020-11-16T22:34:50.464 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2020-11-16T22:34:50.464 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:34:50.464 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T22:34:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is set 2020-11-16T22:34:50.467 INFO:tasks.workunit.client.0.smithi138.stdout:All 50000 operations completed A-OK! 2020-11-16T22:34:50.467 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-16T22:34:50.467 INFO:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-16T22:34:50.468 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:50.469 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:50.480+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:50.566 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:50.581+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.3:76992 3.2 e202/183 3:590b566d:::200.00000018:head v 202'7773, mlcod=202'7771)", 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2020-11-16T22:34:50.621731+0000", 2020-11-16T22:34:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0024951830000000002, 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025187069999999998, 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2020-11-16T22:34:50.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.621731+0000", 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.621731+0000", 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035443300000000003 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622085+0000", 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622095+0000", 2020-11-16T22:34:50.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1516e-05 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622107+0000", 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2949999999999994e-06 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2020-11-16T22:34:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622115+0000", 2020-11-16T22:34:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0634999999999998e-05 2020-11-16T22:34:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2020-11-16T22:34:50.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622176+0000", 2020-11-16T22:34:50.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8418000000000001e-05 2020-11-16T22:34:50.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2020-11-16T22:34:50.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2020-11-16T22:34:50.622204+0000", 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00047359799999999999 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:50.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2020-11-16T22:34:50.622 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2020-11-16T22:34:50.622 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2020-11-16T22:34:50.622 INFO:teuthology.orchestra.run.smithi122.stdout:} 2020-11-16T22:34:50.623 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:50.666 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:50.681+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:50.672 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1: {} 2020-11-16T22:34:50.692 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2: {} 2020-11-16T22:34:50.692 INFO:teuthology.orchestra.run.smithi138.stderr:Error ENXIO: problem getting command descriptions from osd.0 2020-11-16T22:34:50.693 INFO:teuthology.orchestra.run.smithi138.stderr:osd.0: problem getting command descriptions from osd.0 2020-11-16T22:34:50.693 INFO:teuthology.orchestra.run.smithi138.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T22:34:50.693 INFO:teuthology.orchestra.run.smithi138.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T22:34:50.695 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-11-16T22:34:50.695 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-11-16T22:34:50.767 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:50.782+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:50.791 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:50.791 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:50.800 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:50.868 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:50.883+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:50.932 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:50.976 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:50.991+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.070 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.070 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.071 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:51.085+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:51.079 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:51.171 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:51.185+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:51.245 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.245 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.254 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:51.298 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.313+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.372 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:51.387+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:51.386 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.386 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.395 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:51.474 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.489+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.527 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.527 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.536 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:51.575 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.590+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.676 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.691+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.679 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.679 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.689 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:51.776 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:51.791+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:51.838 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.838 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.847 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2020-11-16T22:34:51.848 INFO:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2020-11-16T22:34:51.849 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T22:34:51.893 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.908+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.976 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:51.976 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:51.978 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:51.993+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:51.985 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T22:34:52.031 DEBUG:teuthology.parallel:result is None 2020-11-16T22:34:52.032 INFO:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2020-11-16T22:34:52.064 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2020-11-16T22:34:52.064 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2020-11-16T22:34:52.076 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2020-11-16T22:34:52.076 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2020-11-16T22:34:52.077 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:34:52.077 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:34:52.080 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:52.094+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:52.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[23770]: fuse finished with error 0 and tester_r 0 2020-11-16T22:34:52.143 INFO:teuthology.orchestra.run:waiting for 900 2020-11-16T22:34:52.150 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:52.150 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:52.159 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T22:34:52.200 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:52.215+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:52.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2020-11-16T22:34:52.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:52.206 INFO:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:34:52.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:52.220 INFO:teuthology.orchestra.run.smithi138:> ip netns list 2020-11-16T22:34:52.274 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:34:52.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:52.274 INFO:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-16T22:34:52.282 INFO:tasks.ceph.osd.2.smithi122.stderr:2020-11-16T22:34:52.296+0000 7f5a1c787700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 2957) UID: 0 2020-11-16T22:34:52.324 INFO:teuthology.orchestra.run.smithi194.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:52.324 INFO:teuthology.orchestra.run.smithi194.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:52.333 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T22:34:52.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:34:52.350 INFO:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2020-11-16T22:34:52.383 INFO:tasks.ceph.osd.1.smithi194.stderr:2020-11-16T22:34:52.398+0000 7f57c87e4700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20792) UID: 0 2020-11-16T22:34:52.438 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2020-11-16T22:34:52.448 INFO:tasks.thrashosds:joining thrashosds 2020-11-16T22:34:52.460 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:52.460 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:52.468 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T22:34:52.469 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T22:34:52.604 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:52.604 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:52.617 INFO:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T22:34:52.785 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2020-11-16T22:34:52.785 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T22:34:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:noscrub is unset 2020-11-16T22:34:52.975 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T22:34:53.648 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~215s 2020-11-16T22:34:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:nodeep-scrub is unset 2020-11-16T22:34:59.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~221s 2020-11-16T22:35:05.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~227s 2020-11-16T22:35:11.070 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~233s 2020-11-16T22:35:16.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~238s 2020-11-16T22:35:22.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~244s 2020-11-16T22:35:28.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2020-11-16T22:35:34.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~256s 2020-11-16T22:35:37.142 DEBUG:teuthology.orchestra.console:expect before: b'\r\n[Enter `^Ec?\' for help]\r\n\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H\x1b[0m\x1b[2;30;40m\x1b[0m\x1b[2;37;40m t XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[02;00HInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent XE v2.1.60 \x1b[03;00HPXE 2.1 Build 088 (WfM 2.0) \x1b[04;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu. \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[05;39H\x1b[01;00H \x1b[05;00HPress Ctrl+S to enter the Setup Menu.. \x1b[05;39H\x1b[05;00H \x1b[06;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[02;00HInitializing Intel(R) Boot Agent GE v1.5.13 \x1b[03;00HPXE 2.1 Build 092 (WfM 2.0) \x1b[04;00HPress Ctrl+S to enter the Setup Menu. \x1b[05;00H \x1b[06;00H \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[04;39H\x1b[01;00H \x1b[04;00HPress Ctrl+S to enter the Setup Menu.. \x1b[04;39H\x1b[04;00H \x1b[05;01H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x1b[1;1H\x80\x08 \x08\x1b[2J\x1b[1;1H\x1b[2J\x1b[1;1H\x80\x08 \x08\x80\x08 \x08\x1b[02;00H \x1b[03;00HIntel(R) Boot Agent XE v2.1.60 \x1b[04;00HCopyright (C) 1997-2009, Intel Corporation \x1b[05;00H \x1b[06;00HCLIENT MAC ADDR: 0C C4 7A 88 72 59 GUID: 00000000 0000 0000 0000 0CC47AD9386A \x1b[07;00HDHCP./ \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[14;00H \x1b[15;00H \x1b[16;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[24;00H \x1b[25;00H \x1b[25;00H\x1b[07;06H\x1b[01;00H \x1b[07;00HDHCP.| \x1b[07;06H\x1b[07;00HDHCP./ \x1b[07;06H\x1b[07;00HDHCP.- \x1b[07;06H\x1b[07;00HDHCP.\\ \x1b[07;06H\x1b[07;00HDHCP.| \x1b[07;06H\x1b[07;00HDHCP./ \x1b[07;06H\x1b[07;00HCLIENT IP: 172.21.15.173 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 \x1b[08;00HGATEWAY IP: 172.21.15.254 \x1b[09;00HPXE->EB: !PXE at 96D3:0070, entry point at 96D3:0106 \x1b[10;00H UNDI code segment 96D3:3E70, data segment 8F94:73F0 (574-619kB) \x1b[11;00H UNDI device is PCI 03:00.1, type DIX+802.3 \x1b[12;00H 574kB free base memory after PXE unload \x1b[13;00HiPXE initialising devices... \x1b[13;29H\x1b[13;00HiPXE initialising devices...ok \x1b[17;00H\x1b[0m\x1b[1;37;40miPXE 1.0.0+ (1b67a)\x1b[0m\x1b[2;37;40m -- Open Source Network Boot Firmware -- \x1b[0m\x1b[2;36;40mhttp://ipxe.org\x1b[0m\x1b[2;37;40m \x1b[18;00HFeatures: DNS HTTP HTTPS iSCSI TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT \x1b[19;00HConfiguring (net0 0c:c4:7a:88:72:59)... \x1b[19;40H\x1b[19;00HConfiguring (net0 0c:c4:7a:88:72:59).... \x1b[19;41H\x1b[19;00HConfiguring (net0 0c:c4:7a:88:72:59)..... \x1b[19;42H\x1b[19;00HConfiguring (net0 0c:c4:7a:88:72:59)...... ok \x1b[20;00HReceived DHCP a\x1b[H\x1b[J\x1b[1;1H\x1b[?25l\x1b[m\x1b[H\x1b[J\x1b[1;1H\x1b[20;7H\x1b[mUse the ^ and v keys to change the selection. \n\r Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[4;80H \x1b[7m\x1b[4;1H CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[m\x1b[4;79H\x1b[m\x1b[m\x1b[5;1H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[m\x1b[5;79H\x1b[m\x1b[m\x1b[6;1H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[m\x1b[6;79H\x1b[m\x1b[m\x1b[7;1H \x1b[m\x1b[7;79H\x1b[m\x1b[m\x1b[8;1H \x1b[m\x1b[8;79H\x1b[m\x1b[m\x1b[9;1H \x1b[m\x1b[9;79H\x1b[m\x1b[m\x1b[10;1H \x1b[m\x1b[10;79H\x1b[m\x1b[m\x1b[11;1H \x1b[m\x1b[11;79H\x1b[m\x1b[m\x1b[12;1H \x1b[m\x1b[12;79H\x1b[m\x1b[m\x1b[13;1H \x1b[m\x1b[13;79H\x1b[m\x1b[m\x1b[14;1H \x1b[m\x1b[14;79H\x1b[m\x1b[m\x1b[15;1H \x1b[m\x1b[15;79H\x1b[m\x1b[m\x1b[16;1H \x1b[m\x1b[16;79H\x1b[m\x1b[m\x1b[17;1H \x1b[m\x1b[17;79H\x1b[m\x1b[m\x1b[18;1H \x1b[m\x1b[18;79H\x1b[m\x1b[18;80H \x1b[4;79H\x1b[22;1H The selected entry will be started automatically in 5s. \x1b[4;79H\x1b[03;00H \x1b[04;00H\x1b[0m\x1b[2;30;47m CentOS Linux (4.18.0-193.28.1.el8_2.x86_64) 8 (Core) \x1b[0m\x1b[2;37;40m \x00\x1b[05;00H CentOS Linux (4.18.0-147.el8.x86_64) 8 (Core) \x1b[06;00H CentOS Linux (0-rescue-558cda909eee45cd892f671baf08ec7a) 8 (Core) \x1b[07;00H \x1b[08;00H \x1b[09;00H \x1b[10;00H \x1b[11;00H \x1b[12;00H \x1b[13;00H \x1b[17;00H \x1b[18;00H \x1b[19;00H \x1b[20;00H \x1b[22;1H The selected entry will be started automatically in 4s. \x1b[4;79H\x1b[21;00H Use the * and * keys to change the selection. \x1b[22;00H Press \'e\' to edit the selected item, or \'c\' for a command prompt. \x1b[23;00H The selected entry will be started automatically in 4s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 3s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 3s. \x1b[04;79H\x1b[23;00H The selected\x1b[22;1H The selected entry will be started automatically in 2s. \x1b[4;79H entry will be started automatically in 2s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 1s. \x1b[4;79H\x1b[23;00H The selected entry will be started automatically in 1s. \x1b[04;79H\x1b[22;1H The selected entry will be started automatically in 0s. \x1b[4;79H\x1b[?25h\x1b[H\x1b[J\x1b[1;1H\x1b[H\x1b[J\x1b[1;1H\x1b[04;00H \x1b[05;00H \x1b[06;00H \x1b[21;00H \x1b[22;00H \x1b[23;00H \x1b[01;01H[ 0.000000] Linux version 4.18.0-193.28.1.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Thu Oct 22 00:20:22 UTC 2020\r\n[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: \'x87 floating point registers\'\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: \'SSE registers\'\r\n[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: \'AVX registers\'\r\n[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256\r\n[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using \'standard\' format.\r\n[ 0.000000] BIOS-provided physical RAM map:\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bbff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078ecdfff] usable\r\n[ 0.000000] BIOS-e820: [mem 0x0000000078f7d000-0x00000000798acfff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000798ad000-0x0000000079d49fff] ACPI NVS\r\n[ 0.000000] BIOS-e820: [mem 0x0000000079d4a000-0x000000008fffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved\r\n[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable\r\n[ 0.000000] NX (Execute Disable) protection: active\r\n[ 0.000000] SMBIOS 3.0 present.\r\n[ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0 12/17/2015\r\n[ 0.000000] tsc: Fast TSC calibration using PIT\r\n[ 0.000000] tsc: Detected 3499.702 MHz processor\r\n[ 0.000000] last_pfn = 0x880000 max_arch_pfn = 0x400000000\r\n[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT \r\n[ 0.000000] last_pfn = 0x78ece max_arch_pfn = 0x400000000\r\n[ 0.000000] found SMP MP-table at [mem 0x000fda20-0x000fda2f]\r\n[ 0.000000] kexec: Reserving the low 1M of memory for crashkernel\r\n[ 0.000000] Using GB pages for direct mapping\r\n[ 0.000000] RAMDISK: [mem 0x34e8b000-0x3673dfff]\r\n[ 0.000000] ACPI: Early table checksum verification disabled\r\n[ 0.000000] ACPI: RSDP 0x00000000000F0580 000024 (v02 SUPERM)\r\n[ 0.000000] ACPI: XSDT 0x00000000798AD098 0000B4 (v01 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FACP 0x00000000798E05B0 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: DSDT 0x00000000798AD1E8 0333C4 (v02 SUPERM SMCI--MB 01072009 INTL 20091013)\r\n[ 0.000000] ACPI: FACS 0x0000000079D48F80 000040\r\n[ 0.000000] ACPI: APIC 0x00000000798E06C0 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FPDT 0x00000000798E0788 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: FIDT 0x00000000798E07D0 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013)\r\n[ 0.000000] ACPI: SPMI 0x00000000798E0870 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000)\r\n[ 0.000000] ACPI: MCFG 0x00000000798E08B0 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097)\r\n[ 0.000000] ACPI: UEFI 0x00000000798E08F0 000042 (v01 SUPERM SMCI--MB 01072009 00000000)\r\n[ 0.000000] ACPI: HPET 0x00000000798E0938 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: WDDT 0x00000000798E0970 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013)\r\n[ 0.000000] ACPI: SSDT 0x00000000798E09B0 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798F7B30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: SSDT 0x00000000798FA180 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: PRAD 0x00000000798FA1E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913)\r\n[ 0.000000] ACPI: DMAR 0x00000000798FA2F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013)\r\n[ 0.000000] ACPI: HEST 0x00000000798FA3B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: BERT 0x00000000798FA630 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: ERST 0x00000000798FA660 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] ACPI: EINJ 0x00000000798FA890 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001)\r\n[ 0.000000] No NUMA configuration found\r\n[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000087fffffff]\r\n[ 0.000000] NODE_DATA(0) allocated [mem 0x87ffd6000-0x87fffffff]\r\n[ 0.000000] Using crashkernel=auto, the size chosen is a best effort estimation.\r\n[ 0.000000] Reserving 160MB of memory at 1760MB for crashkernel (System RAM: 32654MB)\r\n[ 0.000000] Zone ranges:\r\n[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]\r\n[ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]\r\n[ 0.000000] Normal [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Device empty\r\n[ 0.000000] Movable zone start for each node\r\n[ 0.000000] Early memory node ranges\r\n[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000008afff]\r\n[ 0.000000] node 0: [mem 0x0000000000100000-0x0000000078ecdfff]\r\n[ 0.000000] node 0: [mem 0x0000000100000000-0x000000087fffffff]\r\n[ 0.000000] Zeroed struct page in unavailable ranges: 29096 pages\r\n[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff]\r\n[ 0.000000] ACPI: PM-Timer IO Port: 0x408\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])\r\n[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])\r\n[ 0.000000] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23\r\n[ 0.000000] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)\r\n[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)\r\n[ 0.000000] Using ACPI (MADT) for SMP configuration information\r\n[ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000\r\n[ 0.000000] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0xb000020 (or later)\r\n[ 0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x0008b000-0x000dffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78ece000-0x78f7cfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x78f7d000-0x798acfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x798ad000-0x79d49fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x79d4a000-0x8fffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0x90000000-0xfed1bfff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed44fff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xfed45000-0xfeffffff]\r\n[ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]\r\n[ 0.000000] [mem 0x90000000-0xfed1bfff] available for PCI devices\r\n[ 0.000000] Booting paravirtualized kernel on bare hardware\r\n[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns\r\n[ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1\r\n[ 0.000000] percpu: Embedded 53 pages/cpu s180224 r8192 d28672 u262144\r\n[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 8228754\r\n[ 0.000000] Policy zone: Normal\r\n[ 0.000000] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64 root=UUID=9bdf28e7-5b1a-4776-884c-191bd8ad39f5 ro crashkernel=auto console=ttyS1,115200 console=tty0 console=ttyS1,115200\r\n[ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://access.redhat.com/ecosystem.\r\n[ 0.000000] Memory: 1856620K/33438048K available (12292K kernel code, 2152K rwdata, 3852K rodata, 2408K init, 6468K bss, 816892K reserved, 0K cma-reserved)\r\n[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1\r\n[ 0.000000] Kernel/User page tables isolation: enabled\r\n[ 0.000000] ftrace: allocating 36345 entries in 142 pages\r\n[ 0.001000] rcu: Hierarchical RCU implementation.\r\n[ 0.001000] rcu: \tRCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.\r\n[ 0.001000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.\r\n[ 0.001000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8\r\n[ 0.001000] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16\r\n[ 0.001000] random: get_random_bytes called from start_kernel+0x36b/0x55b with crng_init=0\r\n[ 0.001000] Console: colour VGA+ 80x25\r\n[ 0.001000] printk: console [tty0] enabled\r\n[ 0.001000] printk: console [ttyS1] enabled\r\n[ 0.001000] ACPI: Core revision 20190703\r\n[ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns\r\n[ 0.001000] APIC: Switch to symmetric I/O mode setup\r\n[ 0.001003] DMAR: Host address width 46\r\n[ 0.002002] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1\r\n[ 0.003004] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de\r\n[ 0.004001] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff\r\n[ 0.005001] DMAR: ATSR flags: 0x0\r\n[ 0.006001] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0\r\n[ 0.007002] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.008001] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0\r\n[ 0.009001] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000\r\n[ 0.010001] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.\r\n[ 0.010002] DMAR-IR: Use \'intremap=no_x2apic_optout\' to override the BIOS setting.\r\n[ 0.012274] DMAR-IR: Enabled IRQ remapping in xapic mode\r\n[ 0.013000] x2apic: IRQ remapping doesn\'t support X2APIC mode\r\n[ 0.014430] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1\r\n[ 0.019002] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x32723801e4f, max_idle_ns: 440795347324 ns\r\n[ 0.020001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.40 BogoMIPS (lpj=3499702)\r\n[ 0.021001] pid_max: default: 32768 minimum: 301\r\n[ 0.022018] Security Framework initialized\r\n[ 0.023000] Yama: becoming mindful.\r\n[ 0.024002] SELinux: Initializing.\r\n[ 0.029955] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)\r\n[ 0.032674] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)\r\n[ 0.033080] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.034074] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)\r\n[ 0.035190] CPU0: Thermal monitoring enabled (TM1)\r\n[ 0.036036] process: using mwait in idle threads\r\n[ 0.037002] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8\r\n[ 0.038000] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4\r\n[ 0.039001] FEATURE SPEC_CTRL Not Present\r\n[ 0.040000] FEATURE IBPB_SUPPORT Not Present\r\n[ 0.041000] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization\r\n[ 0.042001] Spectre V2 : Mitigation: Full generic retpoline\r\n[ 0.043000] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\r\n[ 0.044000] Speculative Store Bypass: Vulnerable\r\n[ 0.045003] TAA: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.046000] MDS: Vulnerable: Clear CPU buffers attempted, no microcode\r\n[ 0.047218] Freeing SMP alternatives memory: 32K\r\n[ 0.051000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1)\r\n[ 0.051061] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver.\r\n[ 0.052002] ... version: 3\r\n[ 0.053000] ... bit width: 48\r\n[ 0.054000] ... generic registers: 4\r\n[ 0.055000] ... value mask: 0000ffffffffffff\r\n[ 0.056000] ... max period: 00007fffffffffff\r\n[ 0.057000] ... fixed-purpose events: 3\r\n[ 0.058000] ... event mask: 000000070000000f\r\n[ 0.059027] rcu: Hierarchical SRCU implementation.\r\n[ 0.060766] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.\r\n[ 0.061015] smp: Bringing up secondary CPUs ...\r\n[ 0.063012] x86: Booting SMP configuration:\r\n[ 0.064001] .... node #0, CPUs: #1 #2 #3 #4\r\n[ 0.066663] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.\r\n[ 0.068001] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details.\r\n[ 0.069073] #5 #6 #7\r\n[ 0.071138] smp: Brought up 1 node, 8 CPUs\r\n[ 0.073002] smpboot: Max logical packages: 1\r\n[ 0.074001] smpboot: Total of 8 processors activated (55995.23 BogoMIPS)\r\n[ 0.281008] node 0 initialised, 7691134 pages in 205ms\r\n[ 0.287209] devtmpfs: initialized\r\n[ 0.288036] x86/mm: Memory block size: 128MB\r\n[ 0.291023] PM: Registering ACPI NVS region [mem 0x798ad000-0x79d49fff] (4837376 bytes)\r\n[ 0.292081] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns\r\n[ 0.303013] futex hash table entries: 2048 (order: 5, 131072 bytes)\r\n[ 0.309059] pinctrl core: initialized pinctrl subsystem\r\n[ 0.315111] NET: Registered protocol family 16\r\n[ 0.320068] audit: initializing netlink subsys (disabled)\r\n[ 0.326007] audit: type=2000 audit(1605566111.326:1): state=initialized audit_enabled=0 res=1\r\n[ 0.335002] cpuidle: using governor menu\r\n[ 0.339023] ACPI FADT declares the system doesn\'t support PCIe ASPM, so disable it\r\n[ 0.347001] ACPI: bus type PCI registered\r\n[ 0.351001] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5\r\n[ 0.358040] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)\r\n[ 0.368002] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820\r\n[ 0.375009] PCI: Using configuration type 1 for base access\r\n[ 0.382015] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages\r\n[ 0.389002] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages\r\n[ 0.396020] cryptd: max_cpu_qlen set to 1000\r\n[ 0.401055] ACPI: Added _OSI(Module Device)\r\n[ 0.406001] ACPI: Added _OSI(Processor Device)\r\n[ 0.411000] ACPI: Added _OSI(3.0 _SCP Extensions)\r\n[ 0.416000] ACPI: Added _OSI(Processor Aggregator Device)\r\n[ 0.421001] ACPI: Added _OSI(Linux-Dell-Video)\r\n[ 0.426000] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)\r\n[ 0.432000] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)\r\n[ 0.461856] ACPI: 4 ACPI AML tables successfully acquired and loaded\r\n[ 0.470950] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored\r\n[ 0.480085] ACPI: Dynamic OEM Table Load:\r\n[ 0.497581] ACPI: Interpreter enabled\r\n[ 0.502011] ACPI: (supports S0 S4 S5)\r\n[ 0.506001] ACPI: Using IOAPIC for interrupt routing\r\n[ 0.511019] HEST: Enabling Firmware First mode for corrected errors.\r\n[ 0.518072] HEST: Table parsing has been initialized.\r\n[ 0.523002] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug\r\n[ 0.533282] ACPI: Enabled 5 GPEs in block 00 to 3F\r\n[ 0.559203] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff])\r\n[ 0.566003] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.576503] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.585333] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.594001] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 0.603025] PCI host bridge to bus 0000:ff\r\n[ 0.607001] pci_bus 0000:ff: root bus resource [bus ff]\r\n[ 0.613008] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000\r\n[ 0.619037] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100\r\n[ 0.625028] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100\r\n[ 0.632027] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000\r\n[ 0.638028] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000\r\n[ 0.644027] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000\r\n[ 0.652030] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000\r\n[ 0.658027] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000\r\n[ 0.664029] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000\r\n[ 0.671026] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000\r\n[ 0.677027] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000\r\n[ 0.684031] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000\r\n[ 0.690027] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000\r\n[ 0.696027] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000\r\n[ 0.703026] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100\r\n[ 0.709027] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000\r\n[ 0.715027] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100\r\n[ 0.722026] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000\r\n[ 0.728026] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000\r\n[ 0.734018] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100\r\n[ 0.741033] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000\r\n[ 0.747055] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000\r\n[ 0.754037] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000\r\n[ 0.760037] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000\r\n[ 0.766037] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000\r\n[ 0.773037] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000\r\n[ 0.779036] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000\r\n[ 0.785029] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000\r\n[ 0.792030] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000\r\n[ 0.798037] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000\r\n[ 0.805037] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000\r\n[ 0.811036] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000\r\n[ 0.817028] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000\r\n[ 0.824029] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000\r\n[ 0.830028] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000\r\n[ 0.836028] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000\r\n[ 0.843029] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000\r\n[ 0.849037] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000\r\n[ 0.855037] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000\r\n[ 0.862037] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000\r\n[ 0.868040] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000\r\n[ 0.875033] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000\r\n[ 0.881028] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000\r\n[ 0.887029] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000\r\n[ 0.894032] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000\r\n[ 0.900028] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000\r\n[ 0.906028] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000\r\n[ 0.913031] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000\r\n[ 0.919032] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000\r\n[ 0.925028] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000\r\n[ 0.932027] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000\r\n[ 0.938027] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000\r\n[ 0.945019] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000\r\n[ 0.951030] pci 0000:ff:1f.0: [8086:6f88] type 00 class 0x088000\r\n[ 0.957028] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000\r\n[ 0.965578] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])\r\n[ 0.972002] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]\r\n[ 0.982184] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR]\r\n[ 0.990249] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]\r\n[ 0.999002] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration\r\n[ 1.008202] PCI host bridge to bus 0000:00\r\n[ 1.012002] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]\r\n[ 1.019001] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window]\r\n[ 1.026001] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]\r\n[ 1.035001] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window]\r\n[ 1.043000] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window]\r\n[ 1.051000] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window]\r\n[ 1.059001] pci_bus 0000:00: root bus resource [bus 00-fe]\r\n[ 1.065006] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000\r\n[ 1.071102] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400\r\n[ 1.077046] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold\r\n[ 1.084072] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400\r\n[ 1.090034] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold\r\n[ 1.097069] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400\r\n[ 1.103044] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold\r\n[ 1.110069] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000\r\n[ 1.116009] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit]\r\n[ 1.123087] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000\r\n[ 1.130008] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit]\r\n[ 1.137086] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000\r\n[ 1.143008] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit]\r\n[ 1.150043] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000\r\n[ 1.157008] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit]\r\n[ 1.164086] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000\r\n[ 1.170008] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit]\r\n[ 1.177087] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000\r\n[ 1.184008] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit]\r\n[ 1.191085] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000\r\n[ 1.197008] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit]\r\n[ 1.204087] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000\r\n[ 1.211008] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit]\r\n[ 1.218084] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000\r\n[ 1.224081] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000\r\n[ 1.231092] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000\r\n[ 1.237041] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020\r\n[ 1.244006] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff]\r\n[ 1.250073] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000\r\n[ 1.257164] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601\r\n[ 1.263016] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117]\r\n[ 1.269006] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103]\r\n[ 1.275006] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7]\r\n[ 1.281006] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3]\r\n[ 1.287006] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f]\r\n[ 1.293006] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff]\r\n[ 1.299035] pci 0000:00:11.4: PME# supported from D3hot\r\n[ 1.305072] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330\r\n[ 1.311017] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit]\r\n[ 1.318050] pci 0000:00:14.0: PME# supported from D3hot D3cold\r\n[ 1.325065] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000\r\n[ 1.331017] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit]\r\n[ 1.338051] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold\r\n[ 1.345065] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000\r\n[ 1.351017] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit]\r\n[ 1.366051] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold\r\n[ 1.372074] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320\r\n[ 1.379018] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff]\r\n[ 1.385071] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold\r\n[ 1.392070] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400\r\n[ 1.398067] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold\r\n[ 1.405073] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400\r\n[ 1.411068] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold\r\n[ 1.418073] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400\r\n[ 1.424068] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold\r\n[ 1.431077] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320\r\n[ 1.437018] pci 0000:00:1d.0: reg 0x10: [mem 0xfb433000-0xfb4333ff]\r\n[ 1.444072] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold\r\n[ 1.450071] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100\r\n[ 1.456153] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601\r\n[ 1.463013] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077]\r\n[ 1.469005] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063]\r\n[ 1.475005] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057]\r\n[ 1.481005] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043]\r\n[ 1.487005] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f]\r\n[ 1.492006] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff]\r\n[ 1.499029] pci 0000:00:1f.2: PME# supported from D3hot\r\n[ 1.505063] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500\r\n[ 1.511015] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit]\r\n[ 1.518017] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f]\r\n[ 1.524149] acpiphp: Slot [1] registered\r\n[ 1.529013] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 1.534106] pci 0000:02:00.0: [8086:0953] type 00 class 0x010802\r\n[ 1.540012] pci 0000:02:00.0: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit]\r\n[ 1.547016] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref]\r\n[ 1.554003] pci 0000:02:00.0: enabling Extended Tags\r\n[ 1.560074] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 1.565002] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 1.572080] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000\r\n[ 1.579013] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref]\r\n[ 1.587004] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f]\r\n[ 1.592010] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref]\r\n[ 1.600004] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref]\r\n[ 1.607035] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.614016] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.622001] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.633008] pci 0000:03:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.640001] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.651117] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000\r\n[ 1.658013] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref]\r\n[ 1.666004] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f]\r\n[ 1.671010] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref]\r\n[ 1.679004] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref]\r\n[ 1.686037] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.693014] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.701001] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs)\r\n[ 1.712008] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.719001] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs)\r\n[ 1.730115] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 1.736001] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 1.742001] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 1.749002] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 1.758028] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 1.763065] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400\r\n[ 1.769071] pci 0000:05:00.0: enabling Extended Tags\r\n[ 1.775064] pci 0000:05:00.0: supports D1 D2\r\n[ 1.776001] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.777076] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 1.778002] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 1.779002] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.780047] pci_bus 0000:06: extended config space not accessible\r\n[ 1.781026] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000\r\n[ 1.782023] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff]\r\n[ 1.783012] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff]\r\n[ 1.784012] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f]\r\n[ 1.786075] pci 0000:06:00.0: supports D1 D2\r\n[ 1.787000] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold\r\n[ 1.788091] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 1.789007] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 1.790003] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 1.791066] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000\r\n[ 1.792029] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff]\r\n[ 1.793018] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f]\r\n[ 1.794009] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff]\r\n[ 1.795114] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold\r\n[ 1.796035] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.797001] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.798021] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.799001] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 1.800099] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000\r\n[ 1.801027] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff]\r\n[ 1.802018] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f]\r\n[ 1.803009] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff]\r\n[ 1.804110] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold\r\n[ 1.805034] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.806001] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs)\r\n[ 1.807021] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref]\r\n[ 1.808001] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs)\r\n[ 1.809119] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 1.810002] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 1.811002] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 1.812004] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08\r\n[ 1.814069] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)\r\n[ 1.815031] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15)\r\n[ 1.816030] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)\r\n[ 1.817030] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)\r\n[ 1.818030] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.819030] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.820029] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.821030] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.\r\n[ 1.822081] iommu: Default domain type: Passthrough \r\n[ 1.828018] pci 0000:06:00.0: vgaarb: setting as boot VGA device\r\n[ 1.829000] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none\r\n[ 1.843001] pci 0000:06:00.0: vgaarb: bridge control possible\r\n[ 1.849000] vgaarb: loaded\r\n[ 1.852057] SCSI subsystem initialized\r\n[ 1.856013] ACPI: bus type USB registered\r\n[ 1.861010] usbcore: registered new interface driver usbfs\r\n[ 1.866005] usbcore: registered new interface driver hub\r\n[ 1.872015] usbcore: registered new device driver usb\r\n[ 1.877018] pps_core: LinuxPPS API ver. 1 registered\r\n[ 1.883000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti \r\n[ 1.893001] PTP clock support registered\r\n[ 1.897014] EDAC MC: Ver: 3.0.0\r\n[ 1.900098] PCI: Using ACPI for IRQ routing\r\n[ 1.910000] NetLabel: Initializing\r\n[ 1.914001] NetLabel: domain hash size = 128\r\n[ 1.919000] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO\r\n[ 1.925009] NetLabel: unlabeled traffic allowed by default\r\n[ 1.931034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0\r\n[ 1.937001] hpet0: 8 comparators, 64-bit 14.318180 MHz counter\r\n[ 1.945019] clocksource: Switched to clocksource tsc-early\r\n[ 1.959270] VFS: Disk quotas dquot_6.6.0\r\n[ 1.963541] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)\r\n[ 1.970789] pnp: PnP ACPI init\r\n[ 1.974472] system 00:01: [io 0x0500-0x057f] has been reserved\r\n[ 1.980719] system 00:01: [io 0x0400-0x047f] has been reserved\r\n[ 1.986968] system 00:01: [io 0x0580-0x059f] has been reserved\r\n[ 1.993219] system 00:01: [io 0x0600-0x061f] has been reserved\r\n[ 1.999466] system 00:01: [io 0x0880-0x0883] has been reserved\r\n[ 2.005717] system 00:01: [io 0x0800-0x081f] has been reserved\r\n[ 2.011965] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved\r\n[ 2.018910] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved\r\n[ 2.025853] system 00:01: [mem 0xff000000-0xffffffff] has been reserved\r\n[ 2.032795] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved\r\n[ 2.039737] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved\r\n[ 2.046682] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved\r\n[ 2.053623] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved\r\n[ 2.060705] system 00:02: [io 0x0a00-0x0a0f] has been reserved\r\n[ 2.066955] system 00:02: [io 0x0a10-0x0a1f] has been reserved\r\n[ 2.073202] system 00:02: [io 0x0a20-0x0a2f] has been reserved\r\n[ 2.079451] system 00:02: [io 0x0a30-0x0a3f] has been reserved\r\n[ 2.085702] system 00:02: [io 0x0a40-0x0a4f] has been reserved\r\n[ 2.092608] pnp: PnP ACPI: found 5 devices\r\n[ 2.102336] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns\r\n[ 2.111833] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000\r\n[ 2.131577] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.140006] pci 0000:00:01.0: PCI bridge to [bus 01]\r\n[ 2.145302] pci 0000:00:02.0: PCI bridge to [bus 02]\r\n[ 2.150593] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff]\r\n[ 2.157720] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.165885] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.174397] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.182647] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.191249] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.199412] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.207925] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.216174] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.224774] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.232938] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.241451] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.249700] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.258301] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.266464] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.274976] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref]\r\n[ 2.283227] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref]\r\n[ 2.291828] pci 0000:00:03.0: PCI bridge to [bus 03]\r\n[ 2.297120] pci 0000:00:03.0: bridge window [io 0xe000-0xefff]\r\n[ 2.303544] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff]\r\n[ 2.310661] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.319002] pci 0000:00:1c.0: PCI bridge to [bus 04]\r\n[ 2.324298] pci 0000:05:00.0: PCI bridge to [bus 06]\r\n[ 2.329591] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff]\r\n[ 2.336017] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.343148] pci 0000:00:1c.2: PCI bridge to [bus 05-06]\r\n[ 2.348700] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff]\r\n[ 2.355127] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff]\r\n[ 2.362246] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref]\r\n[ 2.370593] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref]\r\n[ 2.379025] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref]\r\n[ 2.387364] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref]\r\n[ 2.395789] pci 0000:00:1c.4: PCI bridge to [bus 07]\r\n[ 2.401079] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]\r\n[ 2.407505] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff]\r\n[ 2.414623] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.422962] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc\r\n[ 2.432599] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]\r\n[ 2.439107] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window]\r\n[ 2.445614] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]\r\n[ 2.452820] pci_bus 0000:00: resource 7 [mem 0xfedb0000-0xfedb000f window]\r\n[ 2.460022] pci_bus 0000:00: resource 8 [mem 0xfedc0000-0xfedc000f window]\r\n[ 2.467226] pci_bus 0000:00: resource 9 [mem 0x90000000-0xfbffbfff window]\r\n[ 2.474427] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff]\r\n[ 2.481023] pci_bus 0000:03: resource 0 [io 0xe000-0xefff]\r\n[ 2.486924] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff]\r\n[ 2.493521] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref]\r\n[ 2.501338] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff]\r\n[ 2.507240] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.513838] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff]\r\n[ 2.519738] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff]\r\n[ 2.526336] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff]\r\n[ 2.532238] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff]\r\n[ 2.538835] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref]\r\n[ 2.546748] NET: Registered protocol family 2\r\n[ 2.551534] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes)\r\n[ 2.560214] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)\r\n[ 2.568458] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)\r\n[ 2.575602] TCP: Hash tables configured (established 262144 bind 65536)\r\n[ 2.582588] UDP hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.589164] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)\r\n[ 2.596212] NET: Registered protocol family 1\r\n[ 2.600904] NET: Registered protocol family 44\r\n[ 2.622134] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x6c5 took 15864 usecs\r\n[ 2.642072] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x6c5 took 11651 usecs\r\n[ 2.650077] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes\r\n[ 2.656856] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]\r\n[ 2.665830] Unpacking initramfs...\r\n[ 2.926328] Freeing initrd memory: 25292K\r\n[ 2.939019] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\r\n[ 2.945791] software IO TLB: mapped [mem 0x6a000000-0x6e000000] (64MB)\r\n[ 2.961513] Initialise system trusted keyrings\r\n[ 2.966294] Key type blacklist registered\r\n[ 2.970653] workingset: timestamp_bits=36 max_order=23 bucket_order=0\r\n[ 2.978378] zbud: loaded\r\n[ 2.981506] pstore: using deflate compression\r\n[ 2.986369] Platform Keyring initialized\r\n[ 3.048978] NET: Registered protocol family 38\r\n[ 3.053750] Key type asymmetric registered\r\n[ 3.058176] Asymmetric key parser \'x509\' registered\r\n[ 3.063387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)\r\n[ 3.071405] io scheduler mq-deadline registered\r\n[ 3.076267] io scheduler kyber registered\r\n[ 3.080633] io scheduler bfq registered\r\n[ 3.084843] atomic64_test: passed for x86-64 platform with CX8 and with SSE\r\n[ 3.092389] pcieport 0000:00:01.0: Signaling with IRQ 25\r\n[ 3.098198] pcieport 0000:00:02.0: Signaling with IRQ 27\r\n[ 3.103986] pcieport 0000:00:03.0: Signaling with IRQ 29\r\n[ 3.109750] pcieport 0000:00:1c.0: Signaling with IRQ 30\r\n[ 3.115522] pcieport 0000:00:1c.2: Signaling with IRQ 31\r\n[ 3.121287] pcieport 0000:00:1c.4: Signaling with IRQ 32\r\n[ 3.127013] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4\r\n[ 3.134578] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0\r\n[ 3.143531] ACPI: Power Button [PWRB]\r\n[ 3.147541] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1\r\n[ 3.155541] ACPI: Power Button [PWRF]\r\n[ 3.160575] ERST: Error Record Serialization Table (ERST) support is initialized.\r\n[ 3.168682] pstore: Registered erst as persistent store backend\r\n[ 3.175082] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.\r\n[ 3.183157] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled\r\n[ 3.210334] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A\r\n[ 3.238853] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A\r\n[ 3.247056] Non-volatile memory driver v1.3\r\n[ 3.251968] rdac: device handler registered\r\n[ 3.256525] hp_sw: device handler registered\r\n[ 3.261123] emc: device handler registered\r\n[ 3.265585] alua: device handler registered\r\n[ 3.270125] libphy: Fixed MDIO Bus: probed\r\n[ 3.274604] ehci_hcd: USB 2.0 \'Enhanced\' Host Controller (EHCI) Driver\r\n[ 3.281463] ehci-pci: EHCI PCI platform driver\r\n[ 3.286314] ehci-pci 0000:00:1a.0: EHCI Host Controller\r\n[ 3.291925] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1\r\n[ 3.299923] ehci-pci 0000:00:1a.0: debug port 2\r\n[ 3.308667] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported\r\n[ 3.315792] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000\r\n[ 3.328059] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00\r\n[ 3.334167] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.343027] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.350843] usb usb1: Product: EHCI Host Controller\r\n[ 3.356051] usb usb1: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.363958] usb usb1: SerialNumber: 0000:00:1a.0\r\n[ 3.368997] hub 1-0:1.0: USB hub found\r\n[ 3.373075] hub 1-0:1.0: 2 ports detected\r\n[ 3.377564] ehci-pci 0000:00:1d.0: EHCI Host Controller\r\n[ 3.383173] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2\r\n[ 3.391173] ehci-pci 0000:00:1d.0: debug port 2\r\n[ 3.399926] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported\r\n[ 3.407048] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000\r\n[ 3.420062] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00\r\n[ 3.426165] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.435027] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.442841] usb usb2: Product: EHCI Host Controller\r\n[ 3.455709] usb usb2: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 ehci_hcd\r\n[ 3.463618] usb usb2: SerialNumber: 0000:00:1d.0\r\n[ 3.468646] hub 2-0:1.0: USB hub found\r\n[ 3.472726] hub 2-0:1.0: 2 ports detected\r\n[ 3.477151] ohci_hcd: USB 1.1 \'Open\' Host Controller (OHCI) Driver\r\n[ 3.483662] ohci-pci: OHCI PCI platform driver\r\n[ 3.488440] uhci_hcd: USB Universal Host Controller Interface driver\r\n[ 3.495214] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.500819] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3\r\n[ 3.509862] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810\r\n[ 3.519594] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported\r\n[ 3.526830] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.18\r\n[ 3.535693] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.543506] usb usb3: Product: xHCI Host Controller\r\n[ 3.548714] usb usb3: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.556622] usb usb3: SerialNumber: 0000:00:14.0\r\n[ 3.561646] hub 3-0:1.0: USB hub found\r\n[ 3.565738] hub 3-0:1.0: 15 ports detected\r\n[ 3.571638] xhci_hcd 0000:00:14.0: xHCI Host Controller\r\n[ 3.577251] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4\r\n[ 3.585245] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed\r\n[ 3.591942] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 4.18\r\n[ 3.600801] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1\r\n[ 3.608618] usb usb4: Product: xHCI Host Controller\r\n[ 3.613824] usb usb4: Manufacturer: Linux 4.18.0-193.28.1.el8_2.x86_64 xhci-hcd\r\n[ 3.621733] usb usb4: SerialNumber: 0000:00:14.0\r\n[ 3.626761] hub 4-0:1.0: USB hub found\r\n[ 3.630853] hub 4-0:1.0: 6 ports detected\r\n[ 3.635932] usbcore: registered new interface driver usbserial_generic\r\n[ 3.642795] usbserial: USB Serial support registered for generic\r\n[ 3.649157] i8042: PNP: No PS/2 controller found.\r\n[ 3.654240] mousedev: PS/2 mouse device common for all mice\r\n[ 3.660197] rtc_cmos 00:00: RTC can wake from S4\r\n[ 3.665268] rtc_cmos 00:00: registered as rtc0\r\n[ 3.670052] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram, hpet irqs\r\n[ 3.678314] intel_pstate: Intel P-state driver initializing\r\n[ 3.685014] hidraw: raw HID events driver (C) Jiri Kosina\r\n[ 3.690764] usbcore: registered new interface driver usbhid\r\n[ 3.696663] usbhid: USB HID core driver\r\n[ 3.700881] drop_monitor: Initializing network drop monitor service\r\n[ 3.707524] Initializing XFRM netlink socket\r\n[ 3.712203] NET: Registered protocol family 10\r\n[ 3.717186] Segment Routing with IPv6\r\n[ 3.721193] NET: Registered protocol family 17\r\n[ 3.726033] mpls_gso: MPLS GSO support\r\n[ 3.730463] core: Using 22 MCE banks\r\n[ 3.734385] microcode: sig=0x406f1, pf=0x1, revision=0xb000010\r\n[ 3.740580] usb 1-1: new high-speed USB device number 2 using ehci-pci\r\n[ 3.740632] microcode: Microcode Update Driver: v2.2.\r\n[ 3.747555] resctrl: L3 allocation detected\r\n[ 3.757454] resctrl: L3DATA allocation detected\r\n[ 3.762310] resctrl: L3CODE allocation detected\r\n[ 3.767175] resctrl: L3 monitoring detected\r\n[ 3.771693] AVX2 version of gcm_enc/dec engaged.\r\n[ 3.776652] AES CTR mode by8 optimization enabled\r\n[ 3.806380] sched_clock: Marking stable (3806375404, 0)->(5630464532, -1824089128)\r\n[ 3.814776] registered taskstats version 1\r\n[ 3.819226] Loading compiled-in X.509 certificates\r\n[ 3.843028] usb 2-1: new high-speed USB device number 2 using ehci-pci\r\n[ 3.851862] Loaded X.509 cert \'CentOS kernel signing key: dbeb60232b9048c59969054d0225be669c9ec38b\'\r\n[ 3.861556] Loaded X.509 cert \'CentOS Linux Driver update signing key: 29bd4c0d06d2e9911044b5dc973309139b51d6d5\'\r\n[ 3.872359] Loaded X.509 cert \'CentOS Linux kpatch signing key: b49f086205909dc4da2cfa99376fb191d2f09e78\'\r\n[ 3.880366] usb 1-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05\r\n[ 3.882568] zswap: loaded using pool lzo/zbud\r\n[ 3.891349] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 3.891351] usb 3-12: new high-speed USB device number 2 using xhci_hcd\r\n[ 3.910630] hub 1-1:1.0: USB hub found\r\n[ 3.915118] hub 1-1:1.0: 6 ports detected\r\n[ 3.915218] Key type big_key registered\r\n[ 3.925917] Key type encrypted registered\r\n[ 3.930277] ima: No TPM chip found, activating TPM-bypass!\r\n[ 3.936127] ima: Allocated hash algorithm: sha1\r\n[ 3.940991] evm: Initialising EVM extended attributes:\r\n[ 3.946478] evm: security.selinux\r\n[ 3.950141] evm: security.ima\r\n[ 3.953449] evm: security.capability\r\n[ 3.957376] evm: HMAC attrs: 0x1\r\n[ 3.961345] rtc_cmos 00:00: setting system clock to 2020-11-16 22:35:17 UTC (1605566117)\r\n[ 3.970832] Freeing unused decrypted memory: 2040K\r\n[ 3.976302] Freeing unused kernel memory: 2408K\r\n[ 3.980375] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05\r\n[ 3.989953] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 3.997660] hub 2-1:1.0: USB hub found\r\n[ 4.000037] tsc: Refined TSC clocksource calibration: 3499.994 MHz\r\n[ 4.000042] Write protecting the kernel read-only data: 18432k\r\n[ 4.014432] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734b4358f, max_idle_ns: 440795367814 ns\r\n[ 4.014452] hub 2-1:1.0: 8 ports detected\r\n[ 4.025263] Freeing unused kernel memory: 2020K\r\n[ 4.034258] clocksource: Switched to clocksource tsc\r\n[ 4.039614] Freeing unused kernel memory: 244K\r\n[ 4.046233] usb 3-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00\r\n[ 4.055103] usb 3-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[ 4.055748] hub 3-12:1.0: USB hub found\r\n[ 4.067175] hub 3-12:1.0: 4 ports detected\r\n[ 4.071709] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 4.106158] systemd[1]: Detected architecture x86-64.\r\n[ 4.111550] systemd[1]: Running in initial RAM disk.\r\n\r\nWelcome to \x1b[0;34mCentOS Linux 8 (Core) dracut-049-27.git20190906.el8 (Initramfs)\x1b[0m!\r\n\r\n[ 4.128112] systemd[1]: Set hostname to .\r\n[ 4.163779] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.170793] systemd[1]: Listening on Journal Socket (/dev/log).\r\n[\x1b[0;32m OK \x1b[0m] Listening on Journal Socket (/dev/log).\r\n[ 4.184157] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.191185] systemd[1]: Listening on Journal Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on Journal Socket.\r\n[ 4.203114] random: systemd: uninitialized urandom read (16 bytes read)\r\n[ 4.210791] systemd[1]: Starting Create list of required static device nodes for the current kernel...\r\n Starting Create list of required st\xe2\x80\xa6ce nodes for the current kernel...\r\n[ 4.231149] systemd[1]: Listening on udev Control Socket.\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\n[ 4.244102] systemd[1]: Reached target Timers.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\n[ 4.254548] systemd[1]: Started Hardware RNG Entropy Gatherer Daemon.\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\n Starting Journal Service...\r\n Startin[\x1b[0;32m OK \x1b[ev Kernel Socket.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\n Starting Create Volatile Files and Directories...\r\n Starting Apply Kernel Variables...\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directories.\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\n Starting Create Static Device Nodes in /dev...\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\n[ 4.366008] usb 3-12.1: new low-speed USB device number 3 using xhci_hcd\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\n Starting dracut cmdline hook...\r\n[\x1b[0;32m OK \x1b[0m] Started dracut cmdline hook.\r\n Starting dracut pre-udev hook...\r\n[ 4.471188] usb 3-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00\r\n[ 4.480233] usb 3-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-udev hook.\r\n Starting udev Kernel Device Manager...\r\n[ 4.505846] i HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.0/0003:0557:2419.0001/input/input2\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\n[ 4.584109] hneric 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0\r\n[ 4.609663] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.1/0003:0557:2419.0002/input/input3\r\n[ 4.634056] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\n Starting udev Coldplug all Devices...\r\n Mounting Kernel Configuration File System...\r\n[\x1b[0;32m OK \x1b[0m] Mounted Kernel Configuration File System.\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\n Starting dracut initqueue hook...\r\n Starting Show Plymouth Boot Screen...\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\n[\x1b[0;32m OK \x1b[0m] Started Show Plymouth Boot Screen.\r\r\n[\x1b[0;32m OK \x1b[0m] Rea[ 4.943434] dca service started, version 1.12.1\r\nched target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Password Requests to Plymouth Directory Watch.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Basic System.\r\r\n[ 4.981336] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode\r\n[ 4.990037] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 5.009926] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k\r\n[ 5.011116] nvme nvme0: pci function 0000:02:00.0\r\n[ 5.017237] igb: Copyright (c) 2007-2014 Intel Corporation.\r\n[ 5.031461] scsi host0: ahci\r\n[ 5.035156] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k-rh8.2.0\r\n[ 5.035179] scsi host1: ahci\r\n[ 5.044111] ixgbe: Copyright (c) 1999-2016 Intel Corporation.\r\n[ 5.094704] [drm] Using P2A bridge for configuration\r\n[ 5.100021] [drm] AST 2400 detected\r\n[ 5.103855] [drm] Analog VGA only\r\n[ 5.107517] [drm] dram MCLK=408 Mhz type=1 bus_width=16 size=01000000\r\n[ 5.116938] scsi host2: ahci\r\n[ 5.120176] igb 0000:07:00.0: added PHC on eth0\r\n[ 5.125050] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.128624] nvme nvme0: 8/0/0 default/read/poll queues\r\n[ 5.132270] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d9:38:6a\r\n[ 5.145286] igb 0000:07:00.0: eth0: PBA No: 050B00-000\r\n[ 5.150771] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)\r\n[ 5.177674] [TTM] Zone kernel: Available graphics memory: 16326596 KiB\r\n[ 5.184644] [TTM] Zone dma32: Available graphics memory: 2097152 KiB\r\n[ 5.191512] [TTM] Initializing pool allocator\r\n[ 5.202017] scsi host3: ahci\r\n[ 5.205324] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 34\r\n[ 5.220995] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 34\r\n[ 5.229005] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 34\r\n[ 5.237009] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 34\r\n[ 5.252012] [TTM] Initializing DMA pool allocator\r\n[ 5.269979] fbcon: astdrmfb (fb0) is primary device\r\n[ 5.355731] Console: switching to colour frame buffer device 128x48\r\n[ 5.414694] ast 0000:06:00.0: fb0: astdrmfb frame buffer device\r\n[ 5.430918] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode\r\n[ 5.439125] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst \r\n[ 5.459529] [drm] Initialized ast 0.1.0 20120228 for 0000:06:00.0 on minor 0\r\n[ 5.492435] igb 0000:07:00.1: added PHC on eth1\r\n[ 5.497005] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection\r\n[ 5.503916] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:000:07:00.1: eth1: PBA No: 050B00-000\r\n[ 5.516337] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)\r\n[ 5.525198] scsi host4: ahci\r\n[ 5.529228] scsi host5: ahci\r\n[ 5.533875] scsi host6: ahci\r\n[ 5.542947] scsi host7: ahci\r\n[ 5.547317] scsi host8: ahci\r\n[ 5.551287] ata3: SATA link down (SStatus 0 SControl 300)\r\n[ 5.556738] ata1: SATA link down (SStatus 0 SControl 300)\r\n[ 5.556739] scsi host9: ahci\r\n[ 5.556806] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 63\r\n[ 5.562238] ata2: SATA link down (SStatus 0 SControl 300)\r\n[ 5.565066] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 63\r\n[ 5.565067] ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 63\r\n[ 5.572526] ata4: SATA link down (SStatus 0 SControl 300)\r\n[ 5.577925] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 63\r\n[ 5.607212] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 63\r\n[ 5.615088] aATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 63\r\n[ 5.625237] igb 0000:07:00.1 eno2: renamed from eth1\r\n[ 5.644535] igb 0000:07:00.0 eno1: renamed from eth0\r\n[ 5.925068] ata10: SATA link down (SStatus 0 SControl 300)\r\n[ 5.931389] ata7: SATA link down (SStatus 0 SControl 300)\r\n[ 5.937617] ata8: SATA link down (SStatus 0 SControl 300)\r\n[ 5.943468] ata9: SATA link down (SStatus 0 SControl 300)\r\n[ 5.949639] ata6: SATA link down (SStatus 0 SControl 300)\r\n[ 5.955787] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300)\r\n[ 5.965884] random: crng init done\r\n[ 5.969706] random: 7 urandom warning(s) missed due to ratelimiting\r\n[ 6.031016] ata5.00: NCQ Send/Recv Log not supported\r\n[ 6.036393] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133\r\n[ 6.043060] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 32), AA\r\n[ 6.051959] ata5.00: NCQ Send/Recv Log not supported\r\n[ 6.057324] ata5.00: configured for UDMA/133\r\n[ 6.062104] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5\r\n[ 6.076415] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)\r\n[ 6.084501] sd 4:0:0:0: [sda] Write Protect is off\r\n[ 6.089718] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn\'t support DPO or FUA\r\n[ 6.116338] sda: sda1\r\n[ 6.119298] sd 4:0:0:0: [sda] Attached SCSI disk\r\n[ 6.237391] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.248084] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.256567] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF\r\n[ 6.263168] ixgbe 0000:03:00.0: 0c:c4:7a:88:72:58\r\n[ 6.266095] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection\r\n[\x1b[0;32m OK \x1b[0m] Found device ST1000NM0033-9ZM173 1.\r\r\n[ 6.275942] libphy: ixgbe-mdio: probed\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root Device.\r\r\n[ 6.462136] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0\r\n[ 6.472815] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)\r\n[ 6.481120] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF\r\n[ 6.488560] ixgbe 0000:03:00.1: 0c:c4:7a:88:72:59\r\n[ 6.496655] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection\r\n[ 6.503613] libphy: ixgbe-mdio: probed\r\n[ 6.508610] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0\r\n[ 6.521232] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1\r\n[\x1b[0;32m OK \x1b[0m] Started dracut initqueue hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n Starting File System Check on /dev/\xe2\x80\xa68e7-5b1a-4776-884c-191bd8ad39f5...\r\r\n[\x1b[0;32m OK \x1b[0m] Started File System Check on /dev/d\xe2\x80\xa6f28e7-5b1a-4776-884c-191bd8ad39f5.\r\r\n Mounting /sysroot...\r\r\n[ 6.716481] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)\r\n[\x1b[0;32m OK \x1b[0m] Mounted /sysroot.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Root File System.\r\r\n Starting Reload Configuration from the Real Root...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Reload Configuration from the Real Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Initrd Default Target.\r\r\n Starting dracut pre-pivot and cleanup hook...\r\r\n[\x1b[0;32m OK \x1b[0m] Started dracut pre-pivot and cleanup hook.\r\r\n Starting Cleaning Up and Shutting Down Daemons...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-pivot and cleanup hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Default Target.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Basic System.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Swap.\r\r\n Starting Setup Virtual Console...\r\r\n Stopping udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Volatile Files and Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Local File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Apply Kernel Variables.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut initqueue hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Slices.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Sockets.\r\r\n Starting Plymouth switch root service...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root Device.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped udev Kernel Device Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create Static Device Nodes in /dev.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\r\n Stopping Hardware RNG Entropy Gatherer Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut pre-udev hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped dracut cmdline hook.\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Kernel Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Closed udev Control Socket.\r\r\n Starting Cleanup udevd DB...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped Hardware RNG Entropy Gatherer Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleaning Up and Shutting Down Daemons.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Cleanup udevd DB.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Switch Root.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Plymouth switch root service.\r\r\n Starting Switch Root...\r\r\n[ 7.240118] systemd-journald[319]: Received SIGTERM from PID 1 (systemd).\r\n[ 7.322888] printk: systemd: 21 output lines suppressed due to ratelimiting\r\n[ 8.159106] SELinux: policy capability network_peer_controls=1\r\n[ 8.165359] SELinux: policy capability open_perms=1\r\n[ 8.170632] SELinux: policy capability extended_socket_class=1\r\n[ 8.176851] SELinux: policy capability always_check_network=0\r\n[ 8.182983] SELinux: policy capability cgroup_seclabel=1\r\n[ 8.188673] SELinux: policy capability nnp_nosuid_transition=1\r\n[ 8.210657] audit: type=1403 audit(1605566121.747:2): auid=4294967295 ses=4294967295 lsm=selinux res=1\r\n[ 8.212254] systemd[1]: Successfully loaded SELinux policy in 438.614ms.\r\n[ 8.427402] systemd[1]: Relabelled /dev, /run and /sys/fs/cgroup in 18.762ms.\r\n[ 8.437036] systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)\r\n[ 8.471096] systemd[1]: Detected architecture x86-64.\r\n\r\r\nWelcome to \x1b[0;31mCentOS Linux 8 (Core)\x1b[0m!\r\r\n\r\r\n[ 8.481879] systemd[1]: Set hostname to .\r\n[ 9.453919] systemd[1]: Stopped Switch Root.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Switch Root.\r\r\n[ 9.458873] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart.\r\n[ 9.458926] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.\r\n[ 9.458943] systemd[1]: Stopped Journal Service.\r\n[ 9.501384] systemd[1]: Starting Journal Service...\r\n[ 9.524796] systemd[1]: Listening on LVM2 poll daemon socket.\r\n[ 9.552292] systemd[1]: Listening on initctl Compatibility Named Pipe.\r\n[\x1b[0;32m OK \x1b[0m] Stopped Journal Service.\r\r\n Starting Journal Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on LVM2 poll daemon socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on initctl Compatibility Named Pipe.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Kernel Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-getty.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6mirror@.service instances at once.\r\r\n Mounting POSIX Message Queue File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Switch Root.\r\r\n Starting Apply Kernel Variables...\r\r\n Mounting Kernel Debug File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on RPCbind Server Activation Socket.\r\r\n Starting Remount Root and Kernel File Systems...\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice User and Session Slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mon@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6adosgw@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mgr@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-osd@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-serial\\x2dgetty.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-sshd\\x2dkeygen.slice.\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd Root File System.\r\r\n Mounting Huge Pages File System...\r\r\n[\x1b[0;32m OK \x1b[0m] Stopped target Initrd File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6ph-mds@.service instances at once.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Slices.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Forward Password Requests to Wall Directory Watch.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Device-mapper event daemon FIFOs.\r\r\n Starting Monitoring of LVM2 mirrors\xe2\x80\xa6ng dmeventd or progress polling...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Swap.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local Encrypted Volumes.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Paths.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on udev Control Socket.\r\r\n Starting udev Coldplug all Devices...\r\r\n Starting Create li[ 9.813049] EXT4-fs (sda1): re-mounted. Opts: (null)\r\nst of required st\xe2\x80\xa6ce nodes for the current kernel...\r\r\n[\x1b[0;32m OK \x1b[0m] Set up automount Arbitrary Executab\xe2\x80\xa6rmats File System Automount Point.\r\r\n Starting Setup Virtual Console...\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on Process Core Dump Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target RPC Port Mapper.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Journal Service.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted POSIX Message Queue File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Apply Kernel Variables.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted Kernel Debug File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Remount Root and Kernel File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted Huge Pages File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create list of required sta\xe2\x80\xa6vice nodes for the current kernel.\r\r\n [ 9.918388] systemd-journald[1450]: Received request to flush runtime journal from PID 1\r\n Starting Load/Save Random Seed...\r\r\n Starting Create Static Device Nodes in /dev...\r\r\n Starting Flush Journal to Persistent Storage...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load/Save Random Seed.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Flush Journal to Persistent Storage.\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Coldplug all Devices.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Setup Virtual Console.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Create Static Device Nodes in /dev.\r\r\n Starting udev Kernel Device Manager...\r\r\n[\x1b[0;32m OK \x1b[0m] Started udev Kernel Device Manager.\r\r\n[ 10.495879] ioatdma: Intel(R) QuickData Technology Driver 5.00\r\n[ 10.510201] igb 0000:07:00.0: DCA enabled\r\n[ 10.514684] igb 0000:07:00.1: DCA enabled\r\n[ 10.579979] IPMI message handler: version 39.2\r\n[ 10.586878] ipmi device interface\r\n[ 10.594905] ipmi_si: IPMI System Interface driver\r\n[ 10.600100] ipmi_si dmi-ipmi-si.0: probing via SMBIOS\r\n[ 10.605797] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0\r\n[ 10.613778] ipmi_si: Adding SMBIOS-specified kcs state machine\r\n[ 10.620479] ipmi_si IPI0001:00: probing via ACPI\r\n[ 10.625833] ipmi_si IPI0001:00: [io 0x0ca2] regsize 1 spacing 1 irq 0\r\n[ 10.633393] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI\r\n[ 10.642534] ipmi_si: Adding ACPI-specified kcs state machine\r\n[ 10.649054] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0\r\n[ 10.670718] sd 4:0:0:0: Attached scsi generic sg0 type 0\r\n[\x1b[0;32m OK \x1b[0m] Created slice system-lvm2\\x2dpvscan.slice.\r\r\n[ 10.688427] power_meter ACPI000D:00: Found ACPI power meter.\r\n[ 10.688455] power_meter ACPI000D:00: Ignoring unsafe software power cap!\r\n[ 10.713208] input: PC Speaker as /devices/platform/pcspkr/input/input4\r\n[ 10.713453] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt\r\n[ 10.722075] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed.\r\n[ 10.724196] mei_me 0000:00:16.0: Device doesn\'t have valid ME Interface\r\n[ 10.727529] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer\r\n[ 10.750486] ipmi_si IPI0001:00: Found new BMC (man_id: 0x002a7c, prod_id: 0x0833, dev_id: 0x20)\r\n[ 10.754015] RAPL PMU: hw unit of domain package 2^-14 Joules\r\n[ 10.785923] RAPL PMU: hw unit of domain dram 2^-16 Joules\r\n[ 10.803870] ipmi_si IPI0001:00: IPMI kcs interface initialized\r\n[ 10.823696] ipmi_ssif: IPMI SSIF Interface driver\r\n[ 10.828448] device-mapper: uevent: version 1.0.3\r\n[ 10.828548] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com\r\n Starting LVM event activation on device 259:0...\r\r\n[ 11.108026] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:ff:12.0 (INTERRUPT)\r\n[ 11.119955] EDAC sbridge: Ver: 1.1.2 \r\n[ 11.247192] iTCO_vendor_support: vendor-support=0\r\n[ 11.253205] intel_rapl_common: Found RAPL domain package\r\n[ 11.253208] intel_rapl_common: Found RAPL domain dram\r\n[ 11.253211] intel_rapl_common: DRAM domain energy unit 15300pj\r\n[ 11.255695] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11\r\n[ 11.255803] iTCO_wdt: Found a Wellsburg TCO device (Version=2, TCOBASE=0x0460)\r\n[ 11.256650] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)\r\n[\x1b[0;32m OK \x1b[0m] Found device /dev/vg_nvme/lv_5.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Monitoring of LVM2 mirrors,\xe2\x80\xa6sing dmeventd or progress polling.\r\r\n[\x1b[0;32m OK \x1b[0m] Started LVM event activation on device 259:0.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Local File Systems.\r\r\n Starting Import network configuration from initramfs...\r\r\n Starting Tell Plymouth To Write Out Runtime Data...\r\r\n Starting Restore /run/initramfs on shutdown...\r\r\n Mounting /var/lib/ceph...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Restore /run/initramfs on shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Tell Plymouth To Write Out Runtime Data.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Import network configuration from initramfs.\r\r\n Starting Create Volatile Files and Directories...\r\r\n[ 11.552368] SGI XFS with ACLs, security attributes, no debug enabled\r\n[ 11.561564] XFS (dm-4): Mounting V4 Filesystem\r\n[\x1b[0;32m OK \x1b[0m] Started Create Volatile Files and Directories.\r\r\n[ 11.571759] XFS (dm-4): Ending clean mount\r\n[ 11.613144] RPC: Registered named UNIX socket transport module.\r\n[ 11.613145] RPC: Registered udp transport module.\r\n[ 11.613145] RPC: Registered tcp transport module.\r\n[ 11.613145] RPC: Registered tcp NFSv4.1 backchannel transport module.\r\n Mounting RPC Pipe File System...\r\r\n Starting Security Auditing Service...\r\r\n Starting RPC Bind...\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted /var/lib/ceph.\r\r\n[\x1b[0;32m OK \x1b[0m] Mounted RPC Pipe File System.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target rpc_pipefs.target.\r\r\n[\x1b[0;32m OK \x1b[0m] Started RPC Bind.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Security Auditing Service.\r\r\n Starting Update UTMP about System Boot/Shutdown...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Update UTMP about System Boot/Shutdown.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target System Initialization.\r\r\n[\x1b[0;32m OK \x1b[0m] Started daily update of the root trust anchor for DNSSEC.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on SSSD Kerberos Cache Manager responder socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Daily Cleanup of Temporary Directories.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Generate summary of yesterday\'s process accounting.\r\r\n[\x1b[0;32m OK \x1b[0m] Listening on D-Bus System Message Bus Socket.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Sockets.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Basic System.\r\r\n Starting Load CPU microcode update...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Ceph crash dump collector.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target ceph target allowing\xe2\x80\xa6 ceph*@.service instances at once.\r\r\n Starting Self Monitoring and Reporting Technology (SMART) Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started libstoragemgmt plug-in server daemon.\r\r\n Starting System Security Services Daemon...\r\r\n Starting Kernel Samepage Merging...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Machine Check Exception Logging Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started D-Bus System Message Bus.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hardware RNG Entropy Gatherer Daemon.\r\r\n Starting GSSAPI Proxy Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started irqbalance daemon.\r\r\n Starting NTP client/server...\r\r\n Starting VDO volume services...\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target sshd-keygen.target.\r\r\n Starting Network Manager...\r\r\n Starting Resets System Activity Logs...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Run system activity accounting tool every 10 minutes.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Timers.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Load CPU microcode update.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging.\r\r\n Starting Kernel Samepage Merging (KSM) Tuning Daemon...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Self Monitoring and Reporting Technology (SMART) Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started GSSAPI Proxy Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Resets System Activity Logs.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Kernel Samepage Merging (KSM) Tuning Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target NFS client services.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems (Pre).\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Remote File Systems.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target Network.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Nagios Remote Plugin Executor.\r\r\n Starting OpenSSH server daemon...\r\r\n Starting /etc/rc.d/rc.local Compatibility...\r\r\n Starting Network Manager Wait Online...\r\r\n[\x1b[0;32m OK \x1b[0m] Started NTP client/server.\r\r\n+ \'[\' \'!\' -f /.cephlab_net_configured \']\'\r\n+ set +e\r\n+ attempts=0\r\n+ myips=\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 0 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n Starting Hostname Service...\r\r\n+ myips=\r\n+ attempts=1\r\n+ sleep 1\r\n[\x1b[0;32m OK \x1b[0m] Started OpenSSH server daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Hostname Service.\r\r\n Starting Network Manager Script Dispatcher Service...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Network Manager Script Dispatcher Service.\r\r\n[ 13.843822] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.900738] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready\r\n[ 13.909189] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[ 13.966360] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready\r\n[ 13.974724] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 14.019744] ixgbe 0000:03:00.0: registered PHC device on enp3s0f0\r\n[\x1b[0;32m OK \x1b[0m] Started VDO volume services.\r\r\n[\x1b[0;32m OK \x1b[0m] Started System Security Services Daemon.\r\r\n[\x1b[0;32m OK \x1b[0m] Reached target User and Group Name Lookups.\r\r\n Starting Login Service...\r\r\n Starting Permit User Sessions...\r\r\n[\x1b[0;32m OK \x1b[0m] Started Permit User Sessions.\r\r\n[ 14.129337] IPv6: ADDRCONF(NETDEV_UP): enp3s0f0: link is not ready\r\n[ 14.131789] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n[\x1b[0;32m OK \x1b[0m] Started Command Scheduler.\r\r\n[\x1b[0;32m OK \x1b[0m] Started Job spooling tools.\r\r\n[ 14.169116] ixgbe 0000:03:00.1: registered PHC device on enp3s0f1\r\n[ 14.272879] IPv6: ADDRCONF(NETDEV_UP): enp3s0f1: link is not ready\r\n[\x1b[0;32m OK \x1b[0m] Started Login Service.\r\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 1 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=2\r\n+ sleep 1\r\n[ 14.396475] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4\r\n[ 14.592145] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX\r\n[ 14.600321] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0f1: link becomes ready\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 2 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=\r\n+ attempts=3\r\n+ sleep 1\r\n+ \'[\' \'\' \'!=\' \'\' \']\'\r\n+ \'[\' 3 -ge 10 \']\'\r\n++ ip -4 addr\r\n++ grep -oP \'(?<=inet\\s)\\d+(\\.\\d+){3}\'\r\n++ grep -v \'127.0.0.1\\|127.0.1.1\'\r\n+ myips=172.21.15.173\r\n+ attempts=4\r\n+ sleep 1\r\n+ \'[\' 172.21.15.173 \'!=\' \'\' \']\'\r\n+ set -e\r\n+ \'[\' -n 172.21.15.173 \']\'\r\n+ for ip in $myips\r\n+ timeout 1s ping -I 172.21.15.173 -nq -c1 172.21.0.1\r\n++ dig +short -x 172.21.15.173 @172.21.0.1\r\n++ sed \'s/\\.com.*/\\.com/g\'\r\n+ newhostname=smithi173.front.sepia.ceph.com\r\n+ \'[\' -n smithi173.front.sepia.ceph.com \']\'\r\n+ hostname smithi173.front.sepia.ceph.com\r\n++ hostname -d\r\n+ newdomain=front.sepia.ceph.com\r\n++ hostname -s\r\n+ shorthostname=smithi173\r\n+ echo smithi173\r\n+ grep -q front.sepia.ceph.com /etc/hosts\r\n+ sed -i \'1i172.21.15.173 smithi173.front.sepia.ceph.com smithi173\\\' /etc/hosts\r\n+ break\r\n+ command -v zypper\r\n+ command -v apt-get\r\n+ \'[\' -e /.cephlab_rc_local \']\'\r\n+ exit 0\r\n[\x1b[0;32m OK \x1b[0m] Started /etc/rc.d/rc.local Compatibility.\r\r\n Starting Terminate Plymouth Boot Screen...\r\r\n Starting Hold until boot process finishes up...\r\r\n\r\r\nCentOS Linux 8 (Core)\r\nKernel 4.18.0-193.28.1.el8_2.x86_64 on an x86_64\r\n\r\nActivate the web console with: systemctl enable --now cockpit.socket\r\n\r\n' 2020-11-16T22:35:38.366 DEBUG:teuthology.orchestra.console:expect after: b'smithi173 login: ' 2020-11-16T22:35:38.519 INFO:teuthology.misc:Re-opening connections... 2020-11-16T22:35:38.519 INFO:teuthology.misc:trying to connect to ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:35:38.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:35:38.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:35:38.988 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-16T22:35:39.438 DEBUG:teuthology.misc:waited 0.9186363220214844 2020-11-16T22:35:40.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~262s 2020-11-16T22:35:40.439 DEBUG:tasks.ceph_manager:Mounting data for osd.0 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:35:40.439 INFO:tasks.ceph_manager:Mounting osd.0: dev: ubuntu@smithi173.front.sepia.ceph.com, cluster: cephmountpoint: /var/lib/ceph/osd/ceph-0, type: xfs, options: ['noatime'] 2020-11-16T22:35:40.440 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-16T22:35:40.926 INFO:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:35:41.230 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-16T22:35:41.230 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-16T22:35:41.248 INFO:tasks.ceph.osd.0:Started 2020-11-16T22:35:41.248 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:35:42.312 INFO:teuthology.orchestra.run.smithi173.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2020-11-16T22:35:42.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-16T22:35:42.315 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2020-11-16T22:35:43.522 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:35:43.533+0000 7fd85a1c2ec0 -1 Falling back to public interface 2020-11-16T22:35:46.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2020-11-16T22:35:47.224 INFO:tasks.ceph.osd.0.smithi173.stderr:2020-11-16T22:35:47.238+0000 7fd85a1c2ec0 -1 osd.0 140 log_to_monitors {default=true} 2020-11-16T22:35:47.315 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T22:35:47.444 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2020-11-16T22:35:47.444 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2020-11-16T22:35:47.444 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2020-11-16T22:35:47.445 INFO:teuthology.orchestra.run.smithi173.stdout:} 2020-11-16T22:35:47.482 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2020-11-16T22:35:47.971 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2020-11-16T22:35:47.982 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2020-11-16T22:35:48.962 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2020-11-16T22:35:48.976 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2020-11-16T22:35:49.974 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2020-11-16T22:35:49.988 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-11-16T22:35:49.988 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:35:50.283 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:35:50.285 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:35:50.295 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2020-11-16T22:35:49.984295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6804,"num_bytes_recovered":139054382,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":35,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":715132,"kb_used_data":168932,"kb_used_omap":117,"kb_used_meta":546058,"kb_avail":282400388,"statfs":{"total":289910292480,"available":289177997312,"internally_reserved":0,"allocated":172986368,"data_stored":233922143,"data_compressed":18024135,"data_compressed_allocated":69128192,"data_compressed_original":138256384,"omap_allocated":119921,"internal_metadata":559164303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.192739"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5185","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.357303+0000","last_change":"2020-11-16T22:32:04.140287+0000","last_active":"2020-11-16T22:35:00.357303+0000","last_peered":"2020-11-16T22:35:00.357303+0000","last_clean":"2020-11-16T22:35:00.357303+0000","last_became_active":"2020-11-16T22:32:04.136650+0000","last_became_peered":"2020-11-16T22:32:04.136650+0000","last_unstale":"2020-11-16T22:35:00.357303+0000","last_undegraded":"2020-11-16T22:35:00.357303+0000","last_fullsized":"2020-11-16T22:35:00.357303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"70","reported_epoch":"203","state":"active+undersized","last_fresh":"2020-11-16T22:34:52.979667+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:34:52.979667+0000","last_peered":"2020-11-16T22:34:52.979667+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:34:52.979667+0000","last_undegraded":"2020-11-16T22:34:52.979667+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5552","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.852234+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:34:58.852234+0000","last_peered":"2020-11-16T22:34:58.852234+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:34:58.852234+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17709","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.343591+0000","last_change":"2020-11-16T22:32:38.960533+0000","last_active":"2020-11-16T22:35:00.343591+0000","last_peered":"2020-11-16T22:35:00.343591+0000","last_clean":"2020-11-16T22:35:00.343591+0000","last_became_active":"2020-11-16T22:32:38.951761+0000","last_became_peered":"2020-11-16T22:32:38.951761+0000","last_unstale":"2020-11-16T22:35:00.343591+0000","last_undegraded":"2020-11-16T22:35:00.343591+0000","last_fullsized":"2020-11-16T22:35:00.343591+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"133","reported_epoch":"203","state":"active+undersized","last_fresh":"2020-11-16T22:34:52.978694+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:34:52.978694+0000","last_peered":"2020-11-16T22:34:52.978694+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:34:52.978694+0000","last_undegraded":"2020-11-16T22:34:52.978694+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4555","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.664023+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:34:58.664023+0000","last_peered":"2020-11-16T22:34:58.664023+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:34:58.664023+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6891","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:35:00.333008+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:35:00.333008+0000","last_peered":"2020-11-16T22:35:00.333008+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:35:00.333008+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"80","reported_epoch":"206","state":"active+clean","last_fresh":"2020-11-16T22:35:48.982381+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:48.982381+0000","last_peered":"2020-11-16T22:35:48.982381+0000","last_clean":"2020-11-16T22:35:48.982381+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:48.982381+0000","last_undegraded":"2020-11-16T22:35:48.982381+0000","last_fullsized":"2020-11-16T22:35:48.982381+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11044","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:34:59.345009+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:34:59.345009+0000","last_peered":"2020-11-16T22:34:59.345009+0000","last_clean":"2020-11-16T22:34:59.345009+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:34:59.345009+0000","last_undegraded":"2020-11-16T22:34:59.345009+0000","last_fullsized":"2020-11-16T22:34:59.345009+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23980","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:35:00.340519+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:00.340519+0000","last_peered":"2020-11-16T22:35:00.340519+0000","last_clean":"2020-11-16T22:35:00.340519+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:00.340519+0000","last_undegraded":"2020-11-16T22:35:00.340519+0000","last_fullsized":"2020-11-16T22:35:00.340519+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"155","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.979043+0000","last_change":"2020-11-16T22:32:02.128340+0000","last_active":"2020-11-16T22:34:52.979043+0000","last_peered":"2020-11-16T22:34:52.979043+0000","last_clean":"2020-11-16T22:34:52.979043+0000","last_became_active":"2020-11-16T22:32:02.127967+0000","last_became_peered":"2020-11-16T22:32:02.127967+0000","last_unstale":"2020-11-16T22:34:52.979043+0000","last_undegraded":"2020-11-16T22:34:52.979043+0000","last_fullsized":"2020-11-16T22:34:52.979043+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"141","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978518+0000","last_change":"2020-11-16T22:32:39.953425+0000","last_active":"2020-11-16T22:34:52.978518+0000","last_peered":"2020-11-16T22:34:52.978518+0000","last_clean":"2020-11-16T22:34:52.978518+0000","last_became_active":"2020-11-16T22:32:39.952851+0000","last_became_peered":"2020-11-16T22:32:39.952851+0000","last_unstale":"2020-11-16T22:34:52.978518+0000","last_undegraded":"2020-11-16T22:34:52.978518+0000","last_fullsized":"2020-11-16T22:34:52.978518+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7971","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977253+0000","last_change":"2020-11-16T22:35:48.977253+0000","last_active":"2020-11-16T22:35:47.988597+0000","last_peered":"2020-11-16T22:34:59.691417+0000","last_clean":"2020-11-16T22:34:59.691417+0000","last_became_active":"2020-11-16T22:33:23.957142+0000","last_became_peered":"2020-11-16T22:33:23.957142+0000","last_unstale":"2020-11-16T22:35:48.977253+0000","last_undegraded":"2020-11-16T22:35:48.977253+0000","last_fullsized":"2020-11-16T22:35:48.977253+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"8457","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.978252+0000","last_change":"2020-11-16T22:35:48.978252+0000","last_active":"2020-11-16T22:35:47.986336+0000","last_peered":"2020-11-16T22:35:00.351334+0000","last_clean":"2020-11-16T22:35:00.351334+0000","last_became_active":"2020-11-16T22:33:24.959880+0000","last_became_peered":"2020-11-16T22:33:24.959880+0000","last_unstale":"2020-11-16T22:35:48.978252+0000","last_undegraded":"2020-11-16T22:35:48.978252+0000","last_fullsized":"2020-11-16T22:35:48.978252+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"137","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978528+0000","last_change":"2020-11-16T22:32:38.951861+0000","last_active":"2020-11-16T22:34:52.978528+0000","last_peered":"2020-11-16T22:34:52.978528+0000","last_clean":"2020-11-16T22:34:52.978528+0000","last_became_active":"2020-11-16T22:32:38.951148+0000","last_became_peered":"2020-11-16T22:32:38.951148+0000","last_unstale":"2020-11-16T22:34:52.978528+0000","last_undegraded":"2020-11-16T22:34:52.978528+0000","last_fullsized":"2020-11-16T22:34:52.978528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"336","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.805685+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:34:58.805685+0000","last_peered":"2020-11-16T22:34:58.805685+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:34:58.805685+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33950","reported_epoch":"205","state":"active+clean","last_fresh":"2020-11-16T22:35:47.985221+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:47.985221+0000","last_peered":"2020-11-16T22:35:47.985221+0000","last_clean":"2020-11-16T22:35:47.985221+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:47.985221+0000","last_undegraded":"2020-11-16T22:35:47.985221+0000","last_fullsized":"2020-11-16T22:35:47.985221+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"89","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.976830+0000","last_change":"2020-11-16T22:35:48.977361+0000","last_active":"2020-11-16T22:31:25.000878+0000","last_peered":"2020-11-16T22:31:20.927922+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115476+0000","last_became_peered":"2020-11-16T22:31:04.115476+0000","last_unstale":"2020-11-16T22:35:48.976830+0000","last_undegraded":"2020-11-16T22:35:48.976830+0000","last_fullsized":"2020-11-16T22:35:48.976830+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2288","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:59.380736+0000","last_change":"2020-11-16T22:32:39.957132+0000","last_active":"2020-11-16T22:34:59.380736+0000","last_peered":"2020-11-16T22:34:59.380736+0000","last_clean":"2020-11-16T22:34:59.380736+0000","last_became_active":"2020-11-16T22:32:39.955410+0000","last_became_peered":"2020-11-16T22:32:39.955410+0000","last_unstale":"2020-11-16T22:34:59.380736+0000","last_undegraded":"2020-11-16T22:34:59.380736+0000","last_fullsized":"2020-11-16T22:34:59.380736+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27555","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:35:00.347037+0000","last_change":"2020-11-16T22:33:23.961331+0000","last_active":"2020-11-16T22:35:00.347037+0000","last_peered":"2020-11-16T22:35:00.347037+0000","last_clean":"2020-11-16T22:35:00.347037+0000","last_became_active":"2020-11-16T22:33:23.953893+0000","last_became_peered":"2020-11-16T22:33:23.953893+0000","last_unstale":"2020-11-16T22:35:00.347037+0000","last_undegraded":"2020-11-16T22:35:00.347037+0000","last_fullsized":"2020-11-16T22:35:00.347037+0000","mapping_epoch":183,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2293","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977281+0000","last_change":"2020-11-16T22:35:48.977281+0000","last_active":"2020-11-16T22:35:47.987103+0000","last_peered":"2020-11-16T22:34:59.330740+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:35:48.977281+0000","last_undegraded":"2020-11-16T22:35:48.977281+0000","last_fullsized":"2020-11-16T22:35:48.977281+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"93","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.978890+0000","last_change":"2020-11-16T22:35:48.979060+0000","last_active":"2020-11-16T22:32:00.114224+0000","last_peered":"2020-11-16T22:31:56.100192+0000","last_clean":"2020-11-16T22:31:56.100192+0000","last_became_active":"2020-11-16T22:31:25.006214+0000","last_became_peered":"2020-11-16T22:31:25.006214+0000","last_unstale":"2020-11-16T22:35:48.978890+0000","last_undegraded":"2020-11-16T22:35:48.978890+0000","last_fullsized":"2020-11-16T22:35:48.978890+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24199","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.336246+0000","last_change":"2020-11-16T22:32:39.965767+0000","last_active":"2020-11-16T22:35:00.336246+0000","last_peered":"2020-11-16T22:35:00.336246+0000","last_clean":"2020-11-16T22:35:00.336246+0000","last_became_active":"2020-11-16T22:32:39.955272+0000","last_became_peered":"2020-11-16T22:32:39.955272+0000","last_unstale":"2020-11-16T22:35:00.336246+0000","last_undegraded":"2020-11-16T22:35:00.336246+0000","last_fullsized":"2020-11-16T22:35:00.336246+0000","mapping_epoch":171,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"137","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978914+0000","last_change":"2020-11-16T22:32:38.951920+0000","last_active":"2020-11-16T22:34:52.978914+0000","last_peered":"2020-11-16T22:34:52.978914+0000","last_clean":"2020-11-16T22:34:52.978914+0000","last_became_active":"2020-11-16T22:32:38.951358+0000","last_became_peered":"2020-11-16T22:32:38.951358+0000","last_unstale":"2020-11-16T22:34:52.978914+0000","last_undegraded":"2020-11-16T22:34:52.978914+0000","last_fullsized":"2020-11-16T22:34:52.978914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2360","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:58.764776+0000","last_change":"2020-11-16T22:32:02.132544+0000","last_active":"2020-11-16T22:34:58.764776+0000","last_peered":"2020-11-16T22:34:58.764776+0000","last_clean":"2020-11-16T22:34:58.764776+0000","last_became_active":"2020-11-16T22:32:02.132057+0000","last_became_peered":"2020-11-16T22:32:02.132057+0000","last_unstale":"2020-11-16T22:34:58.764776+0000","last_undegraded":"2020-11-16T22:34:58.764776+0000","last_fullsized":"2020-11-16T22:34:58.764776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11776000,"data_stored":8595981,"data_compressed":1014673,"data_compressed_allocated":1945600,"data_compressed_original":3891200,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":12,"acting":14,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":106481772,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155856896,"data_stored":222969974,"data_compressed":17009462,"data_compressed_allocated":67182592,"data_compressed_original":134365184,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":11,"acting":11,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":1,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":11,"acting":14,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468039,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293500,"kb_used_data":72308,"kb_used_omap":26,"kb_used_meta":221157,"kb_avail":94078340,"statfs":{"total":96636764160,"available":96336220160,"internally_reserved":0,"allocated":74043392,"data_stored":105711636,"data_compressed":8203488,"data_compressed_allocated":32829440,"data_compressed_original":65658880,"omap_allocated":27325,"internal_metadata":226465091},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:35:01 2020","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55800000000000005,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.032,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.008,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.58799999999999997}]}]},{"osd":0,"up_from":0,"seq":1,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":35936,"kb_used_omap":90,"kb_used_meta":21797,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":36798464,"data_stored":56020845,"data_compressed":6807386,"data_compressed_allocated":25092096,"data_compressed_original":50184192,"omap_allocated":92596,"internal_metadata":22320716},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":8,"seq":34359738522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363800,"kb_used_data":60688,"kb_used_omap":0,"kb_used_meta":303104,"kb_avail":94008040,"statfs":{"total":96636764160,"available":96264232960,"internally_reserved":0,"allocated":62144512,"data_stored":72189662,"data_compressed":3013261,"data_compressed_allocated":11206656,"data_compressed_original":22413312,"omap_allocated":0,"internal_metadata":310378496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.0109999999999999}]}]}]}} 2020-11-16T22:35:52.539 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:35:52.838 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:35:52.839 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:35:52.850 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2020-11-16T22:35:52.018779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6804,"num_bytes_recovered":139054382,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":35,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":715132,"kb_used_data":168932,"kb_used_omap":117,"kb_used_meta":546058,"kb_avail":282400388,"statfs":{"total":289910292480,"available":289177997312,"internally_reserved":0,"allocated":172986368,"data_stored":233922143,"data_compressed":18024135,"data_compressed_allocated":69128192,"data_compressed_original":138256384,"omap_allocated":119921,"internal_metadata":559164303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038350"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5185","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.357303+0000","last_change":"2020-11-16T22:32:04.140287+0000","last_active":"2020-11-16T22:35:00.357303+0000","last_peered":"2020-11-16T22:35:00.357303+0000","last_clean":"2020-11-16T22:35:00.357303+0000","last_became_active":"2020-11-16T22:32:04.136650+0000","last_became_peered":"2020-11-16T22:32:04.136650+0000","last_unstale":"2020-11-16T22:35:00.357303+0000","last_undegraded":"2020-11-16T22:35:00.357303+0000","last_fullsized":"2020-11-16T22:35:00.357303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"70","reported_epoch":"203","state":"active+undersized","last_fresh":"2020-11-16T22:34:52.979667+0000","last_change":"2020-11-16T22:31:04.113247+0000","last_active":"2020-11-16T22:34:52.979667+0000","last_peered":"2020-11-16T22:34:52.979667+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:34:52.979667+0000","last_undegraded":"2020-11-16T22:34:52.979667+0000","last_fullsized":"2020-11-16T22:31:04.110687+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5552","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.852234+0000","last_change":"2020-11-16T22:31:04.114002+0000","last_active":"2020-11-16T22:34:58.852234+0000","last_peered":"2020-11-16T22:34:58.852234+0000","last_clean":"2020-11-16T22:31:02.506229+0000","last_became_active":"2020-11-16T22:31:04.114002+0000","last_became_peered":"2020-11-16T22:31:04.114002+0000","last_unstale":"2020-11-16T22:34:58.852234+0000","last_undegraded":"2020-11-16T22:31:04.110628+0000","last_fullsized":"2020-11-16T22:31:04.110456+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16777626,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17709","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.343591+0000","last_change":"2020-11-16T22:32:38.960533+0000","last_active":"2020-11-16T22:35:00.343591+0000","last_peered":"2020-11-16T22:35:00.343591+0000","last_clean":"2020-11-16T22:35:00.343591+0000","last_became_active":"2020-11-16T22:32:38.951761+0000","last_became_peered":"2020-11-16T22:32:38.951761+0000","last_unstale":"2020-11-16T22:35:00.343591+0000","last_undegraded":"2020-11-16T22:35:00.343591+0000","last_fullsized":"2020-11-16T22:35:00.343591+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"133","reported_epoch":"203","state":"active+undersized","last_fresh":"2020-11-16T22:34:52.978694+0000","last_change":"2020-11-16T22:31:04.113966+0000","last_active":"2020-11-16T22:34:52.978694+0000","last_peered":"2020-11-16T22:34:52.978694+0000","last_clean":"2020-11-16T22:31:00.092578+0000","last_became_active":"2020-11-16T22:31:04.113966+0000","last_became_peered":"2020-11-16T22:31:04.113966+0000","last_unstale":"2020-11-16T22:34:52.978694+0000","last_undegraded":"2020-11-16T22:34:52.978694+0000","last_fullsized":"2020-11-16T22:31:04.110420+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4555","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.664023+0000","last_change":"2020-11-16T22:31:04.113964+0000","last_active":"2020-11-16T22:34:58.664023+0000","last_peered":"2020-11-16T22:34:58.664023+0000","last_clean":"2020-11-16T22:31:00.092722+0000","last_became_active":"2020-11-16T22:31:04.113964+0000","last_became_peered":"2020-11-16T22:31:04.113964+0000","last_unstale":"2020-11-16T22:34:58.664023+0000","last_undegraded":"2020-11-16T22:31:04.110693+0000","last_fullsized":"2020-11-16T22:31:04.110569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33554500,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6891","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:35:00.333008+0000","last_change":"2020-11-16T22:31:04.114321+0000","last_active":"2020-11-16T22:35:00.333008+0000","last_peered":"2020-11-16T22:35:00.333008+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:35:00.333008+0000","last_undegraded":"2020-11-16T22:31:04.110975+0000","last_fullsized":"2020-11-16T22:31:04.110880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"80","reported_epoch":"206","state":"active+clean","last_fresh":"2020-11-16T22:35:48.982381+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:48.982381+0000","last_peered":"2020-11-16T22:35:48.982381+0000","last_clean":"2020-11-16T22:35:48.982381+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:48.982381+0000","last_undegraded":"2020-11-16T22:35:48.982381+0000","last_fullsized":"2020-11-16T22:35:48.982381+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11044","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:34:59.345009+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:34:59.345009+0000","last_peered":"2020-11-16T22:34:59.345009+0000","last_clean":"2020-11-16T22:34:59.345009+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:34:59.345009+0000","last_undegraded":"2020-11-16T22:34:59.345009+0000","last_fullsized":"2020-11-16T22:34:59.345009+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23980","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:35:00.340519+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:00.340519+0000","last_peered":"2020-11-16T22:35:00.340519+0000","last_clean":"2020-11-16T22:35:00.340519+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:00.340519+0000","last_undegraded":"2020-11-16T22:35:00.340519+0000","last_fullsized":"2020-11-16T22:35:00.340519+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"155","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.979043+0000","last_change":"2020-11-16T22:32:02.128340+0000","last_active":"2020-11-16T22:34:52.979043+0000","last_peered":"2020-11-16T22:34:52.979043+0000","last_clean":"2020-11-16T22:34:52.979043+0000","last_became_active":"2020-11-16T22:32:02.127967+0000","last_became_peered":"2020-11-16T22:32:02.127967+0000","last_unstale":"2020-11-16T22:34:52.979043+0000","last_undegraded":"2020-11-16T22:34:52.979043+0000","last_fullsized":"2020-11-16T22:34:52.979043+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"141","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978518+0000","last_change":"2020-11-16T22:32:39.953425+0000","last_active":"2020-11-16T22:34:52.978518+0000","last_peered":"2020-11-16T22:34:52.978518+0000","last_clean":"2020-11-16T22:34:52.978518+0000","last_became_active":"2020-11-16T22:32:39.952851+0000","last_became_peered":"2020-11-16T22:32:39.952851+0000","last_unstale":"2020-11-16T22:34:52.978518+0000","last_undegraded":"2020-11-16T22:34:52.978518+0000","last_fullsized":"2020-11-16T22:34:52.978518+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7971","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977253+0000","last_change":"2020-11-16T22:35:48.977253+0000","last_active":"2020-11-16T22:35:47.988597+0000","last_peered":"2020-11-16T22:34:59.691417+0000","last_clean":"2020-11-16T22:34:59.691417+0000","last_became_active":"2020-11-16T22:33:23.957142+0000","last_became_peered":"2020-11-16T22:33:23.957142+0000","last_unstale":"2020-11-16T22:35:48.977253+0000","last_undegraded":"2020-11-16T22:35:48.977253+0000","last_fullsized":"2020-11-16T22:35:48.977253+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"8457","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.978252+0000","last_change":"2020-11-16T22:35:48.978252+0000","last_active":"2020-11-16T22:35:47.986336+0000","last_peered":"2020-11-16T22:35:00.351334+0000","last_clean":"2020-11-16T22:35:00.351334+0000","last_became_active":"2020-11-16T22:33:24.959880+0000","last_became_peered":"2020-11-16T22:33:24.959880+0000","last_unstale":"2020-11-16T22:35:48.978252+0000","last_undegraded":"2020-11-16T22:35:48.978252+0000","last_fullsized":"2020-11-16T22:35:48.978252+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"137","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978528+0000","last_change":"2020-11-16T22:32:38.951861+0000","last_active":"2020-11-16T22:34:52.978528+0000","last_peered":"2020-11-16T22:34:52.978528+0000","last_clean":"2020-11-16T22:34:52.978528+0000","last_became_active":"2020-11-16T22:32:38.951148+0000","last_became_peered":"2020-11-16T22:32:38.951148+0000","last_unstale":"2020-11-16T22:34:52.978528+0000","last_undegraded":"2020-11-16T22:34:52.978528+0000","last_fullsized":"2020-11-16T22:34:52.978528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"336","reported_epoch":"204","state":"active+undersized+degraded","last_fresh":"2020-11-16T22:34:58.805685+0000","last_change":"2020-11-16T22:31:04.113920+0000","last_active":"2020-11-16T22:34:58.805685+0000","last_peered":"2020-11-16T22:34:58.805685+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:34:58.805685+0000","last_undegraded":"2020-11-16T22:31:04.110577+0000","last_fullsized":"2020-11-16T22:31:04.110452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33950","reported_epoch":"205","state":"active+clean","last_fresh":"2020-11-16T22:35:47.985221+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:47.985221+0000","last_peered":"2020-11-16T22:35:47.985221+0000","last_clean":"2020-11-16T22:35:47.985221+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:47.985221+0000","last_undegraded":"2020-11-16T22:35:47.985221+0000","last_fullsized":"2020-11-16T22:35:47.985221+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"89","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.976830+0000","last_change":"2020-11-16T22:35:48.977361+0000","last_active":"2020-11-16T22:31:25.000878+0000","last_peered":"2020-11-16T22:31:20.927922+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115476+0000","last_became_peered":"2020-11-16T22:31:04.115476+0000","last_unstale":"2020-11-16T22:35:48.976830+0000","last_undegraded":"2020-11-16T22:35:48.976830+0000","last_fullsized":"2020-11-16T22:35:48.976830+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2288","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:59.380736+0000","last_change":"2020-11-16T22:32:39.957132+0000","last_active":"2020-11-16T22:34:59.380736+0000","last_peered":"2020-11-16T22:34:59.380736+0000","last_clean":"2020-11-16T22:34:59.380736+0000","last_became_active":"2020-11-16T22:32:39.955410+0000","last_became_peered":"2020-11-16T22:32:39.955410+0000","last_unstale":"2020-11-16T22:34:59.380736+0000","last_undegraded":"2020-11-16T22:34:59.380736+0000","last_fullsized":"2020-11-16T22:34:59.380736+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27555","reported_epoch":"204","state":"active+clean","last_fresh":"2020-11-16T22:35:00.347037+0000","last_change":"2020-11-16T22:33:23.961331+0000","last_active":"2020-11-16T22:35:00.347037+0000","last_peered":"2020-11-16T22:35:00.347037+0000","last_clean":"2020-11-16T22:35:00.347037+0000","last_became_active":"2020-11-16T22:33:23.953893+0000","last_became_peered":"2020-11-16T22:33:23.953893+0000","last_unstale":"2020-11-16T22:35:00.347037+0000","last_undegraded":"2020-11-16T22:35:00.347037+0000","last_fullsized":"2020-11-16T22:35:00.347037+0000","mapping_epoch":183,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2293","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977281+0000","last_change":"2020-11-16T22:35:48.977281+0000","last_active":"2020-11-16T22:35:47.987103+0000","last_peered":"2020-11-16T22:34:59.330740+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:35:48.977281+0000","last_undegraded":"2020-11-16T22:35:48.977281+0000","last_fullsized":"2020-11-16T22:35:48.977281+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"93","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.978890+0000","last_change":"2020-11-16T22:35:48.979060+0000","last_active":"2020-11-16T22:32:00.114224+0000","last_peered":"2020-11-16T22:31:56.100192+0000","last_clean":"2020-11-16T22:31:56.100192+0000","last_became_active":"2020-11-16T22:31:25.006214+0000","last_became_peered":"2020-11-16T22:31:25.006214+0000","last_unstale":"2020-11-16T22:35:48.978890+0000","last_undegraded":"2020-11-16T22:35:48.978890+0000","last_fullsized":"2020-11-16T22:35:48.978890+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24199","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.336246+0000","last_change":"2020-11-16T22:32:39.965767+0000","last_active":"2020-11-16T22:35:00.336246+0000","last_peered":"2020-11-16T22:35:00.336246+0000","last_clean":"2020-11-16T22:35:00.336246+0000","last_became_active":"2020-11-16T22:32:39.955272+0000","last_became_peered":"2020-11-16T22:32:39.955272+0000","last_unstale":"2020-11-16T22:35:00.336246+0000","last_undegraded":"2020-11-16T22:35:00.336246+0000","last_fullsized":"2020-11-16T22:35:00.336246+0000","mapping_epoch":171,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"137","reported_epoch":"203","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:52.978914+0000","last_change":"2020-11-16T22:32:38.951920+0000","last_active":"2020-11-16T22:34:52.978914+0000","last_peered":"2020-11-16T22:34:52.978914+0000","last_clean":"2020-11-16T22:34:52.978914+0000","last_became_active":"2020-11-16T22:32:38.951358+0000","last_became_peered":"2020-11-16T22:32:38.951358+0000","last_unstale":"2020-11-16T22:34:52.978914+0000","last_undegraded":"2020-11-16T22:34:52.978914+0000","last_fullsized":"2020-11-16T22:34:52.978914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2360","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:34:58.764776+0000","last_change":"2020-11-16T22:32:02.132544+0000","last_active":"2020-11-16T22:34:58.764776+0000","last_peered":"2020-11-16T22:34:58.764776+0000","last_clean":"2020-11-16T22:34:58.764776+0000","last_became_active":"2020-11-16T22:32:02.132057+0000","last_became_peered":"2020-11-16T22:32:02.132057+0000","last_unstale":"2020-11-16T22:34:58.764776+0000","last_undegraded":"2020-11-16T22:34:58.764776+0000","last_fullsized":"2020-11-16T22:34:58.764776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6747,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11776000,"data_stored":8595981,"data_compressed":1014673,"data_compressed_allocated":1945600,"data_compressed_original":3891200,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":12,"acting":14,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":106481772,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155856896,"data_stored":222969974,"data_compressed":17009462,"data_compressed_allocated":67182592,"data_compressed_original":134365184,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":11,"acting":11,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":1,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":11,"acting":14,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468039,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293500,"kb_used_data":72308,"kb_used_omap":26,"kb_used_meta":221157,"kb_avail":94078340,"statfs":{"total":96636764160,"available":96336220160,"internally_reserved":0,"allocated":74043392,"data_stored":105711636,"data_compressed":8203488,"data_compressed_allocated":32829440,"data_compressed_original":65658880,"omap_allocated":27325,"internal_metadata":226465091},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Nov 16 22:35:01 2020","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55800000000000005,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.032,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.008,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.58799999999999997}]}]},{"osd":0,"up_from":0,"seq":1,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":35936,"kb_used_omap":90,"kb_used_meta":21797,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":36798464,"data_stored":56020845,"data_compressed":6807386,"data_compressed_allocated":25092096,"data_compressed_original":50184192,"omap_allocated":92596,"internal_metadata":22320716},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":8,"seq":34359738522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363800,"kb_used_data":60688,"kb_used_omap":0,"kb_used_meta":303104,"kb_avail":94008040,"statfs":{"total":96636764160,"available":96264232960,"internally_reserved":0,"allocated":62144512,"data_stored":72189662,"data_compressed":3013261,"data_compressed_allocated":11206656,"data_compressed_original":22413312,"omap_allocated":0,"internal_metadata":310378496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.0109999999999999}]}]}]}} 2020-11-16T22:35:53.667 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:35:54.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:35:54.027 INFO:teuthology.orchestra.run.smithi138.stdout:{"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 2 pgs inactive, 2 pgs peering","count":4},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 339/106 objects degraded (319.811%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":211,"num_osds":3,"num_up_osds":3,"osd_up_since":1605566147,"num_in_osds":3,"osd_in_since":1605565389,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"peering","count":3},{"state_name":"remapped+peering","count":3},{"state_name":"active+clean+remapped","count":2},{"state_name":"active+clean+wait","count":1},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"active+recovery_wait+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":25,"num_pools":4,"num_objects":53,"data_bytes":113220706,"bytes_used":766951424,"bytes_avail":289143341056,"bytes_total":289910292480,"inactive_pgs_ratio":0.23999999463558197,"degraded_objects":339,"degraded_total":106,"degraded_ratio":3.1981132075471699,"misplaced_objects":1018,"misplaced_total":106,"misplaced_ratio":9.6037735849056602,"recovering_objects_per_sec":86,"recovering_bytes_per_sec":5061336,"recovering_keys_per_sec":0,"num_objects_recovered":346,"num_bytes_recovered":20249387,"num_keys_recovered":0},"fsmap":{"epoch":4,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4212}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2020-11-16T22:30:57.726881+0000","services":{}},"progress_events":{}} 2020-11-16T22:35:54.040 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2020-11-16T22:35:54.040 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:35:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:35:54.329 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:35:54.339 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":590,"stamp":"2020-11-16T22:35:53.971437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":339,"num_objects_misplaced":1018,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7150,"num_bytes_recovered":159303769,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":748976,"kb_used_data":191128,"kb_used_omap":117,"kb_used_meta":557706,"kb_avail":282366544,"statfs":{"total":289910292480,"available":289143341056,"internally_reserved":0,"allocated":195715072,"data_stored":279204961,"data_compressed":23681249,"data_compressed_allocated":91754496,"data_compressed_original":183508992,"omap_allocated":119921,"internal_metadata":571091855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":321,"num_objects_misplaced":1003,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":20249387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.987141"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5185","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.357303+0000","last_change":"2020-11-16T22:32:04.140287+0000","last_active":"2020-11-16T22:35:00.357303+0000","last_peered":"2020-11-16T22:35:00.357303+0000","last_clean":"2020-11-16T22:35:00.357303+0000","last_became_active":"2020-11-16T22:32:04.136650+0000","last_became_peered":"2020-11-16T22:32:04.136650+0000","last_unstale":"2020-11-16T22:35:00.357303+0000","last_undegraded":"2020-11-16T22:35:00.357303+0000","last_fullsized":"2020-11-16T22:35:00.357303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"146","reported_epoch":"208","state":"active+clean+wait","last_fresh":"2020-11-16T22:35:51.005225+0000","last_change":"2020-11-16T22:35:48.991389+0000","last_active":"2020-11-16T22:35:51.005225+0000","last_peered":"2020-11-16T22:35:51.005225+0000","last_clean":"2020-11-16T22:35:51.005225+0000","last_became_active":"2020-11-16T22:35:48.988556+0000","last_became_peered":"2020-11-16T22:35:48.988556+0000","last_unstale":"2020-11-16T22:35:51.005225+0000","last_undegraded":"2020-11-16T22:35:51.005225+0000","last_fullsized":"2020-11-16T22:35:51.005225+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5581","reported_epoch":"209","state":"active+clean","last_fresh":"2020-11-16T22:35:52.022222+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:35:52.022222+0000","last_peered":"2020-11-16T22:35:52.022222+0000","last_clean":"2020-11-16T22:35:52.022222+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:35:52.022222+0000","last_undegraded":"2020-11-16T22:35:52.022222+0000","last_fullsized":"2020-11-16T22:35:52.022222+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17724","reported_epoch":"208","state":"active+recovery_wait+remapped","last_fresh":"2020-11-16T22:35:51.150504+0000","last_change":"2020-11-16T22:35:51.150504+0000","last_active":"2020-11-16T22:35:51.150504+0000","last_peered":"2020-11-16T22:35:51.150504+0000","last_clean":"2020-11-16T22:35:00.366509+0000","last_became_active":"2020-11-16T22:35:51.140548+0000","last_became_peered":"2020-11-16T22:35:51.140548+0000","last_unstale":"2020-11-16T22:35:51.150504+0000","last_undegraded":"2020-11-16T22:35:51.150504+0000","last_fullsized":"2020-11-16T22:35:51.150504+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":447,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"146","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006947+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:35:51.006947+0000","last_peered":"2020-11-16T22:35:51.006947+0000","last_clean":"2020-11-16T22:35:51.006947+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:35:51.006947+0000","last_undegraded":"2020-11-16T22:35:51.006947+0000","last_fullsized":"2020-11-16T22:35:51.006947+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4573","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006988+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:35:51.006988+0000","last_peered":"2020-11-16T22:35:51.006988+0000","last_clean":"2020-11-16T22:35:51.006988+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:35:51.006988+0000","last_undegraded":"2020-11-16T22:35:51.006988+0000","last_fullsized":"2020-11-16T22:35:51.006988+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"7244","reported_epoch":"208","state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2020-11-16T22:35:51.153908+0000","last_change":"2020-11-16T22:35:51.151361+0000","last_active":"2020-11-16T22:35:51.153908+0000","last_peered":"2020-11-16T22:35:51.153908+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:35:50.155500+0000","last_became_peered":"2020-11-16T22:35:50.155500+0000","last_unstale":"2020-11-16T22:35:51.153908+0000","last_undegraded":"2020-11-16T22:35:51.147283+0000","last_fullsized":"2020-11-16T22:35:49.987343+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":339,"num_objects_misplaced":571,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":792,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"80","reported_epoch":"206","state":"active+clean","last_fresh":"2020-11-16T22:35:48.982381+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:48.982381+0000","last_peered":"2020-11-16T22:35:48.982381+0000","last_clean":"2020-11-16T22:35:48.982381+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:48.982381+0000","last_undegraded":"2020-11-16T22:35:48.982381+0000","last_fullsized":"2020-11-16T22:35:48.982381+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11049","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006550+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:35:51.006550+0000","last_peered":"2020-11-16T22:35:51.006550+0000","last_clean":"2020-11-16T22:35:51.006550+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:35:51.006550+0000","last_undegraded":"2020-11-16T22:35:51.006550+0000","last_fullsized":"2020-11-16T22:35:51.006550+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23985","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006674+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:51.006674+0000","last_peered":"2020-11-16T22:35:51.006674+0000","last_clean":"2020-11-16T22:35:51.006674+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:51.006674+0000","last_undegraded":"2020-11-16T22:35:51.006674+0000","last_fullsized":"2020-11-16T22:35:51.006674+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"169","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006858+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:35:51.006858+0000","last_peered":"2020-11-16T22:35:51.006858+0000","last_clean":"2020-11-16T22:35:51.006858+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:35:51.006858+0000","last_undegraded":"2020-11-16T22:35:51.006858+0000","last_fullsized":"2020-11-16T22:35:51.006858+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"155","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006373+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:35:51.006373+0000","last_peered":"2020-11-16T22:35:51.006373+0000","last_clean":"2020-11-16T22:35:51.006373+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:35:51.006373+0000","last_undegraded":"2020-11-16T22:35:51.006373+0000","last_fullsized":"2020-11-16T22:35:51.006373+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7971","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977253+0000","last_change":"2020-11-16T22:35:48.977253+0000","last_active":"2020-11-16T22:35:47.988597+0000","last_peered":"2020-11-16T22:34:59.691417+0000","last_clean":"2020-11-16T22:34:59.691417+0000","last_became_active":"2020-11-16T22:33:23.957142+0000","last_became_peered":"2020-11-16T22:33:23.957142+0000","last_unstale":"2020-11-16T22:35:48.977253+0000","last_undegraded":"2020-11-16T22:35:48.977253+0000","last_fullsized":"2020-11-16T22:35:48.977253+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"8457","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.978252+0000","last_change":"2020-11-16T22:35:48.978252+0000","last_active":"2020-11-16T22:35:47.986336+0000","last_peered":"2020-11-16T22:35:00.351334+0000","last_clean":"2020-11-16T22:35:00.351334+0000","last_became_active":"2020-11-16T22:33:24.959880+0000","last_became_peered":"2020-11-16T22:33:24.959880+0000","last_unstale":"2020-11-16T22:35:48.978252+0000","last_undegraded":"2020-11-16T22:35:48.978252+0000","last_fullsized":"2020-11-16T22:35:48.978252+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"151","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006383+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:35:51.006383+0000","last_peered":"2020-11-16T22:35:51.006383+0000","last_clean":"2020-11-16T22:35:51.006383+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:35:51.006383+0000","last_undegraded":"2020-11-16T22:35:51.006383+0000","last_fullsized":"2020-11-16T22:35:51.006383+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"356","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006452+0000","last_change":"2020-11-16T22:35:49.054386+0000","last_active":"2020-11-16T22:35:51.006452+0000","last_peered":"2020-11-16T22:35:51.006452+0000","last_clean":"2020-11-16T22:35:51.006452+0000","last_became_active":"2020-11-16T22:35:48.996346+0000","last_became_peered":"2020-11-16T22:35:48.996346+0000","last_unstale":"2020-11-16T22:35:51.006452+0000","last_undegraded":"2020-11-16T22:35:51.006452+0000","last_fullsized":"2020-11-16T22:35:51.006452+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3949634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33950","reported_epoch":"205","state":"active+clean","last_fresh":"2020-11-16T22:35:47.985221+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:47.985221+0000","last_peered":"2020-11-16T22:35:47.985221+0000","last_clean":"2020-11-16T22:35:47.985221+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:47.985221+0000","last_undegraded":"2020-11-16T22:35:47.985221+0000","last_fullsized":"2020-11-16T22:35:47.985221+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"89","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.976830+0000","last_change":"2020-11-16T22:35:48.977361+0000","last_active":"2020-11-16T22:31:25.000878+0000","last_peered":"2020-11-16T22:31:20.927922+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115476+0000","last_became_peered":"2020-11-16T22:31:04.115476+0000","last_unstale":"2020-11-16T22:35:48.976830+0000","last_undegraded":"2020-11-16T22:35:48.976830+0000","last_fullsized":"2020-11-16T22:35:48.976830+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2299","reported_epoch":"209","state":"peering","last_fresh":"2020-11-16T22:35:52.015627+0000","last_change":"2020-11-16T22:35:52.015627+0000","last_active":"2020-11-16T22:35:47.986335+0000","last_peered":"2020-11-16T22:34:59.558902+0000","last_clean":"2020-11-16T22:34:59.558902+0000","last_became_active":"2020-11-16T22:32:39.955410+0000","last_became_peered":"2020-11-16T22:32:39.955410+0000","last_unstale":"2020-11-16T22:35:52.015627+0000","last_undegraded":"2020-11-16T22:35:52.015627+0000","last_fullsized":"2020-11-16T22:35:52.015627+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"28021","reported_epoch":"209","state":"active+recovering+undersized+remapped","last_fresh":"2020-11-16T22:35:52.254649+0000","last_change":"2020-11-16T22:35:51.239917+0000","last_active":"2020-11-16T22:35:52.254649+0000","last_peered":"2020-11-16T22:35:52.254649+0000","last_clean":"2020-11-16T22:35:00.349182+0000","last_became_active":"2020-11-16T22:35:50.234105+0000","last_became_peered":"2020-11-16T22:35:50.234105+0000","last_unstale":"2020-11-16T22:35:52.254649+0000","last_undegraded":"2020-11-16T22:35:52.254649+0000","last_fullsized":"2020-11-16T22:35:49.987276+0000","mapping_epoch":206,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1660,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2293","reported_epoch":"206","state":"remapped+peering","last_fresh":"2020-11-16T22:35:48.977281+0000","last_change":"2020-11-16T22:35:48.977281+0000","last_active":"2020-11-16T22:35:47.987103+0000","last_peered":"2020-11-16T22:34:59.330740+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:35:48.977281+0000","last_undegraded":"2020-11-16T22:35:48.977281+0000","last_fullsized":"2020-11-16T22:35:48.977281+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"93","reported_epoch":"206","state":"peering","last_fresh":"2020-11-16T22:35:48.978890+0000","last_change":"2020-11-16T22:35:48.979060+0000","last_active":"2020-11-16T22:32:00.114224+0000","last_peered":"2020-11-16T22:31:56.100192+0000","last_clean":"2020-11-16T22:31:56.100192+0000","last_became_active":"2020-11-16T22:31:25.006214+0000","last_became_peered":"2020-11-16T22:31:25.006214+0000","last_unstale":"2020-11-16T22:35:48.978890+0000","last_undegraded":"2020-11-16T22:35:48.978890+0000","last_fullsized":"2020-11-16T22:35:48.978890+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24199","reported_epoch":"204","state":"active+clean+remapped","last_fresh":"2020-11-16T22:35:00.336246+0000","last_change":"2020-11-16T22:32:39.965767+0000","last_active":"2020-11-16T22:35:00.336246+0000","last_peered":"2020-11-16T22:35:00.336246+0000","last_clean":"2020-11-16T22:35:00.336246+0000","last_became_active":"2020-11-16T22:32:39.955272+0000","last_became_peered":"2020-11-16T22:32:39.955272+0000","last_unstale":"2020-11-16T22:35:00.336246+0000","last_undegraded":"2020-11-16T22:35:00.336246+0000","last_fullsized":"2020-11-16T22:35:00.336246+0000","mapping_epoch":171,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"151","reported_epoch":"208","state":"active+clean","last_fresh":"2020-11-16T22:35:51.006982+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:35:51.006982+0000","last_peered":"2020-11-16T22:35:51.006982+0000","last_clean":"2020-11-16T22:35:51.006982+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:35:51.006982+0000","last_undegraded":"2020-11-16T22:35:51.006982+0000","last_fullsized":"2020-11-16T22:35:51.006982+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2387","reported_epoch":"209","state":"active+clean","last_fresh":"2020-11-16T22:35:52.021670+0000","last_change":"2020-11-16T22:35:52.021670+0000","last_active":"2020-11-16T22:35:52.021670+0000","last_peered":"2020-11-16T22:35:52.021670+0000","last_clean":"2020-11-16T22:35:52.021670+0000","last_became_active":"2020-11-16T22:35:52.020590+0000","last_became_peered":"2020-11-16T22:35:52.020590+0000","last_unstale":"2020-11-16T22:35:52.021670+0000","last_undegraded":"2020-11-16T22:35:52.021670+0000","last_fullsized":"2020-11-16T22:35:52.021670+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":16777216,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":339,"num_objects_misplaced":1018,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7085,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9764864,"data_stored":7345256,"data_compressed":936989,"data_compressed_allocated":1662976,"data_compressed_original":3325952,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":14,"acting":13,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":126731159,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178585600,"data_stored":268385095,"data_compressed":22700667,"data_compressed_allocated":89870336,"data_compressed_original":179740672,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":14,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293676,"kb_used_data":72356,"kb_used_omap":26,"kb_used_meta":221285,"kb_avail":94078164,"statfs":{"total":96636764160,"available":96336039936,"internally_reserved":0,"allocated":74092544,"data_stored":105730757,"data_compressed":8203488,"data_compressed_allocated":32829440,"data_compressed_original":65658880,"omap_allocated":27325,"internal_metadata":226596163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:35:01 2020","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55800000000000005,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.032,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.008,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.74099999999999999}]}]},{"osd":0,"up_from":205,"seq":880468295682,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91500,"kb_used_data":58084,"kb_used_omap":90,"kb_used_meta":33317,"kb_avail":94280340,"statfs":{"total":96636764160,"available":96543068160,"internally_reserved":0,"allocated":59478016,"data_stored":101284542,"data_compressed":12464500,"data_compressed_allocated":47718400,"data_compressed_original":95436800,"omap_allocated":92596,"internal_metadata":34117196},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":1,"up_from":8,"seq":34359738522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363800,"kb_used_data":60688,"kb_used_omap":0,"kb_used_meta":303104,"kb_avail":94008040,"statfs":{"total":96636764160,"available":96264232960,"internally_reserved":0,"allocated":62144512,"data_stored":72189662,"data_compressed":3013261,"data_compressed_allocated":11206656,"data_compressed_original":22413312,"omap_allocated":0,"internal_metadata":310378496},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.0109999999999999}]}]}]}} 2020-11-16T22:35:57.952 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:35:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:35:58.258 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:35:58.269 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2020-11-16T22:35:58.038374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8151,"num_bytes_recovered":176080985,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":50,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":726816,"kb_used_data":181960,"kb_used_omap":117,"kb_used_meta":544714,"kb_avail":282388704,"statfs":{"total":289910292480,"available":289166032896,"internally_reserved":0,"allocated":186327040,"data_stored":272316783,"data_compressed":22881060,"data_compressed_allocated":90140672,"data_compressed_original":180281344,"omap_allocated":119921,"internal_metadata":557788047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":-1018,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1001,"num_bytes_recovered":16777216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.066936"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5196","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020885+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:35:53.020885+0000","last_peered":"2020-11-16T22:35:53.020885+0000","last_clean":"2020-11-16T22:35:53.020885+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:35:53.020885+0000","last_undegraded":"2020-11-16T22:35:53.020885+0000","last_fullsized":"2020-11-16T22:35:53.020885+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"151","reported_epoch":"213","state":"active+clean+wait","last_fresh":"2020-11-16T22:35:55.982996+0000","last_change":"2020-11-16T22:35:48.991389+0000","last_active":"2020-11-16T22:35:55.982996+0000","last_peered":"2020-11-16T22:35:55.982996+0000","last_clean":"2020-11-16T22:35:55.982996+0000","last_became_active":"2020-11-16T22:35:48.988556+0000","last_became_peered":"2020-11-16T22:35:48.988556+0000","last_unstale":"2020-11-16T22:35:55.982996+0000","last_undegraded":"2020-11-16T22:35:55.982996+0000","last_fullsized":"2020-11-16T22:35:55.982996+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5586","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984353+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:35:55.984353+0000","last_peered":"2020-11-16T22:35:55.984353+0000","last_clean":"2020-11-16T22:35:55.984353+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:35:55.984353+0000","last_undegraded":"2020-11-16T22:35:55.984353+0000","last_fullsized":"2020-11-16T22:35:55.984353+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17728","reported_epoch":"214","state":"active+clean","last_fresh":"2020-11-16T22:35:57.000134+0000","last_change":"2020-11-16T22:35:57.000134+0000","last_active":"2020-11-16T22:35:57.000134+0000","last_peered":"2020-11-16T22:35:57.000134+0000","last_clean":"2020-11-16T22:35:57.000134+0000","last_became_active":"2020-11-16T22:35:56.993064+0000","last_became_peered":"2020-11-16T22:35:56.993064+0000","last_unstale":"2020-11-16T22:35:57.000134+0000","last_undegraded":"2020-11-16T22:35:57.000134+0000","last_fullsized":"2020-11-16T22:35:57.000134+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"151","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984352+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:35:55.984352+0000","last_peered":"2020-11-16T22:35:55.984352+0000","last_clean":"2020-11-16T22:35:55.984352+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:35:55.984352+0000","last_undegraded":"2020-11-16T22:35:55.984352+0000","last_fullsized":"2020-11-16T22:35:55.984352+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4578","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984503+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:35:55.984503+0000","last_peered":"2020-11-16T22:35:55.984503+0000","last_clean":"2020-11-16T22:35:55.984503+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:35:55.984503+0000","last_undegraded":"2020-11-16T22:35:55.984503+0000","last_fullsized":"2020-11-16T22:35:55.984503+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"8281","reported_epoch":"214","state":"active+recovering+undersized+remapped","last_fresh":"2020-11-16T22:35:57.256293+0000","last_change":"2020-11-16T22:35:57.236890+0000","last_active":"2020-11-16T22:35:57.256293+0000","last_peered":"2020-11-16T22:35:57.256293+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:35:50.155500+0000","last_became_peered":"2020-11-16T22:35:50.155500+0000","last_unstale":"2020-11-16T22:35:57.256293+0000","last_undegraded":"2020-11-16T22:35:57.256293+0000","last_fullsized":"2020-11-16T22:35:49.987343+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1138,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"85","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020640+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:53.020640+0000","last_peered":"2020-11-16T22:35:53.020640+0000","last_clean":"2020-11-16T22:35:53.020640+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:53.020640+0000","last_undegraded":"2020-11-16T22:35:53.020640+0000","last_fullsized":"2020-11-16T22:35:53.020640+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11054","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984489+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:35:55.984489+0000","last_peered":"2020-11-16T22:35:55.984489+0000","last_clean":"2020-11-16T22:35:55.984489+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:35:55.984489+0000","last_undegraded":"2020-11-16T22:35:55.984489+0000","last_fullsized":"2020-11-16T22:35:55.984489+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23990","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984559+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:55.984559+0000","last_peered":"2020-11-16T22:35:55.984559+0000","last_clean":"2020-11-16T22:35:55.984559+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:55.984559+0000","last_undegraded":"2020-11-16T22:35:55.984559+0000","last_fullsized":"2020-11-16T22:35:55.984559+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"174","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984384+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:35:55.984384+0000","last_peered":"2020-11-16T22:35:55.984384+0000","last_clean":"2020-11-16T22:35:55.984384+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:35:55.984384+0000","last_undegraded":"2020-11-16T22:35:55.984384+0000","last_fullsized":"2020-11-16T22:35:55.984384+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"160","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984221+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:35:55.984221+0000","last_peered":"2020-11-16T22:35:55.984221+0000","last_clean":"2020-11-16T22:35:55.984221+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:35:55.984221+0000","last_undegraded":"2020-11-16T22:35:55.984221+0000","last_fullsized":"2020-11-16T22:35:55.984221+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7997","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020455+0000","last_change":"2020-11-16T22:35:52.028496+0000","last_active":"2020-11-16T22:35:53.020455+0000","last_peered":"2020-11-16T22:35:53.020455+0000","last_clean":"2020-11-16T22:35:53.020455+0000","last_became_active":"2020-11-16T22:35:52.025147+0000","last_became_peered":"2020-11-16T22:35:52.025147+0000","last_unstale":"2020-11-16T22:35:53.020455+0000","last_undegraded":"2020-11-16T22:35:53.020455+0000","last_fullsized":"2020-11-16T22:35:53.020455+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":39223632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9345","reported_epoch":"211","state":"active+undersized+remapped","last_fresh":"2020-11-16T22:35:54.076539+0000","last_change":"2020-11-16T22:35:54.076782+0000","last_active":"2020-11-16T22:35:54.076539+0000","last_peered":"2020-11-16T22:35:54.076539+0000","last_clean":"2020-11-16T22:35:00.351334+0000","last_became_active":"2020-11-16T22:35:50.147467+0000","last_became_peered":"2020-11-16T22:35:50.147467+0000","last_unstale":"2020-11-16T22:35:54.076539+0000","last_undegraded":"2020-11-16T22:35:54.076539+0000","last_fullsized":"2020-11-16T22:35:49.988321+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":["1","0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"156","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984226+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:35:55.984226+0000","last_peered":"2020-11-16T22:35:55.984226+0000","last_clean":"2020-11-16T22:35:55.984226+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:35:55.984226+0000","last_undegraded":"2020-11-16T22:35:55.984226+0000","last_fullsized":"2020-11-16T22:35:55.984226+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"361","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984296+0000","last_change":"2020-11-16T22:35:49.054386+0000","last_active":"2020-11-16T22:35:55.984296+0000","last_peered":"2020-11-16T22:35:55.984296+0000","last_clean":"2020-11-16T22:35:55.984296+0000","last_became_active":"2020-11-16T22:35:48.996346+0000","last_became_peered":"2020-11-16T22:35:48.996346+0000","last_unstale":"2020-11-16T22:35:55.984296+0000","last_undegraded":"2020-11-16T22:35:55.984296+0000","last_fullsized":"2020-11-16T22:35:55.984296+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3949634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33955","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020662+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:53.020662+0000","last_peered":"2020-11-16T22:35:53.020662+0000","last_clean":"2020-11-16T22:35:53.020662+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:53.020662+0000","last_undegraded":"2020-11-16T22:35:53.020662+0000","last_fullsized":"2020-11-16T22:35:53.020662+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"101","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020744+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:35:53.020744+0000","last_peered":"2020-11-16T22:35:53.020744+0000","last_clean":"2020-11-16T22:35:53.020744+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:35:53.020744+0000","last_undegraded":"2020-11-16T22:35:53.020744+0000","last_fullsized":"2020-11-16T22:35:53.020744+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2311","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.983123+0000","last_change":"2020-11-16T22:35:53.024437+0000","last_active":"2020-11-16T22:35:55.983123+0000","last_peered":"2020-11-16T22:35:55.983123+0000","last_clean":"2020-11-16T22:35:55.983123+0000","last_became_active":"2020-11-16T22:35:53.023864+0000","last_became_peered":"2020-11-16T22:35:53.023864+0000","last_unstale":"2020-11-16T22:35:55.983123+0000","last_undegraded":"2020-11-16T22:35:55.983123+0000","last_fullsized":"2020-11-16T22:35:55.983123+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"28461","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984353+0000","last_change":"2020-11-16T22:35:54.995752+0000","last_active":"2020-11-16T22:35:55.984353+0000","last_peered":"2020-11-16T22:35:55.984353+0000","last_clean":"2020-11-16T22:35:55.984353+0000","last_became_active":"2020-11-16T22:35:54.988294+0000","last_became_peered":"2020-11-16T22:35:54.988294+0000","last_unstale":"2020-11-16T22:35:55.984353+0000","last_undegraded":"2020-11-16T22:35:55.984353+0000","last_fullsized":"2020-11-16T22:35:55.984353+0000","mapping_epoch":211,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1873,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2315","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020925+0000","last_change":"2020-11-16T22:35:52.023474+0000","last_active":"2020-11-16T22:35:53.020925+0000","last_peered":"2020-11-16T22:35:53.020925+0000","last_clean":"2020-11-16T22:35:53.020925+0000","last_became_active":"2020-11-16T22:35:52.022453+0000","last_became_peered":"2020-11-16T22:35:52.022453+0000","last_unstale":"2020-11-16T22:35:53.020925+0000","last_undegraded":"2020-11-16T22:35:53.020925+0000","last_fullsized":"2020-11-16T22:35:53.020925+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"105","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020820+0000","last_change":"2020-11-16T22:35:50.081501+0000","last_active":"2020-11-16T22:35:53.020820+0000","last_peered":"2020-11-16T22:35:53.020820+0000","last_clean":"2020-11-16T22:35:53.020820+0000","last_became_active":"2020-11-16T22:35:50.080825+0000","last_became_peered":"2020-11-16T22:35:50.080825+0000","last_unstale":"2020-11-16T22:35:53.020820+0000","last_undegraded":"2020-11-16T22:35:53.020820+0000","last_fullsized":"2020-11-16T22:35:53.020820+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24211","reported_epoch":"210","state":"active+clean","last_fresh":"2020-11-16T22:35:53.020791+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:35:53.020791+0000","last_peered":"2020-11-16T22:35:53.020791+0000","last_clean":"2020-11-16T22:35:53.020791+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:35:53.020791+0000","last_undegraded":"2020-11-16T22:35:53.020791+0000","last_fullsized":"2020-11-16T22:35:53.020791+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"156","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984688+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:35:55.984688+0000","last_peered":"2020-11-16T22:35:55.984688+0000","last_clean":"2020-11-16T22:35:55.984688+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:35:55.984688+0000","last_undegraded":"2020-11-16T22:35:55.984688+0000","last_fullsized":"2020-11-16T22:35:55.984688+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2392","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984656+0000","last_change":"2020-11-16T22:35:52.021670+0000","last_active":"2020-11-16T22:35:55.984656+0000","last_peered":"2020-11-16T22:35:55.984656+0000","last_clean":"2020-11-16T22:35:55.984656+0000","last_became_active":"2020-11-16T22:35:52.020590+0000","last_became_peered":"2020-11-16T22:35:52.020590+0000","last_unstale":"2020-11-16T22:35:55.984656+0000","last_undegraded":"2020-11-16T22:35:55.984656+0000","last_fullsized":"2020-11-16T22:35:55.984656+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":16777216,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8082,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":16,"acting":14,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":143508375,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172294144,"data_stored":259996487,"data_compressed":22249761,"data_compressed_allocated":87773184,"data_compressed_original":175546368,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293788,"kb_used_data":72404,"kb_used_omap":26,"kb_used_meta":221349,"kb_avail":94078052,"statfs":{"total":96636764160,"available":96335925248,"internally_reserved":0,"allocated":74141696,"data_stored":105756666,"data_compressed":8203488,"data_compressed_allocated":32829440,"data_compressed_original":65658880,"omap_allocated":27325,"internal_metadata":226661699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Nov 16 22:35:01 2020","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55800000000000005,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.032,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.008,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.69399999999999995}]}]},{"osd":0,"up_from":205,"seq":880468295683,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79992,"kb_used_data":48816,"kb_used_omap":90,"kb_used_meta":31077,"kb_avail":94291848,"statfs":{"total":96636764160,"available":96554852352,"internally_reserved":0,"allocated":49987584,"data_stored":94347790,"data_compressed":11664311,"data_compressed_allocated":46104576,"data_compressed_original":92209152,"omap_allocated":92596,"internal_metadata":31823436},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":1,"up_from":8,"seq":34359738523,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353036,"kb_used_data":60740,"kb_used_omap":0,"kb_used_meta":292288,"kb_avail":94018804,"statfs":{"total":96636764160,"available":96275255296,"internally_reserved":0,"allocated":62197760,"data_stored":72212327,"data_compressed":3013261,"data_compressed_allocated":11206656,"data_compressed_original":22413312,"omap_allocated":0,"internal_metadata":299302912},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.70699999999999996}]}]}]}} 2020-11-16T22:36:00.919 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:36:01.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:01.282 INFO:teuthology.orchestra.run.smithi138.stdout:{"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":777,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":216,"num_osds":3,"num_up_osds":3,"osd_up_since":1605566147,"num_in_osds":3,"osd_in_since":1605565389,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":24},{"state_name":"active+clean+wait","count":1}],"num_pgs":25,"num_pools":4,"num_objects":53,"data_bytes":113220706,"bytes_used":747716608,"bytes_avail":289162575872,"bytes_total":289910292480,"recovering_objects_per_sec":195,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":4,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4212}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2020-11-16T22:30:57.726881+0000","services":{}},"progress_events":{}} 2020-11-16T22:36:01.295 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2020-11-16T22:36:01.295 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:01.582 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:01.584 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:01.594 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2020-11-16T22:35:59.794788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8376,"num_bytes_recovered":176080985,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":730192,"kb_used_data":175864,"kb_used_omap":117,"kb_used_meta":554186,"kb_avail":282385328,"statfs":{"total":289910292480,"available":289162575872,"internally_reserved":0,"allocated":180084736,"data_stored":263953868,"data_compressed":22430154,"data_compressed_allocated":88043520,"data_compressed_original":176087040,"omap_allocated":119921,"internal_metadata":567487375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":-1018,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000763"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5201","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041604+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:35:58.041604+0000","last_peered":"2020-11-16T22:35:58.041604+0000","last_clean":"2020-11-16T22:35:58.041604+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:35:58.041604+0000","last_undegraded":"2020-11-16T22:35:58.041604+0000","last_fullsized":"2020-11-16T22:35:58.041604+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"151","reported_epoch":"213","state":"active+clean+wait","last_fresh":"2020-11-16T22:35:55.982996+0000","last_change":"2020-11-16T22:35:48.991389+0000","last_active":"2020-11-16T22:35:55.982996+0000","last_peered":"2020-11-16T22:35:55.982996+0000","last_clean":"2020-11-16T22:35:55.982996+0000","last_became_active":"2020-11-16T22:35:48.988556+0000","last_became_peered":"2020-11-16T22:35:48.988556+0000","last_unstale":"2020-11-16T22:35:55.982996+0000","last_undegraded":"2020-11-16T22:35:55.982996+0000","last_fullsized":"2020-11-16T22:35:55.982996+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5586","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984353+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:35:55.984353+0000","last_peered":"2020-11-16T22:35:55.984353+0000","last_clean":"2020-11-16T22:35:55.984353+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:35:55.984353+0000","last_undegraded":"2020-11-16T22:35:55.984353+0000","last_fullsized":"2020-11-16T22:35:55.984353+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17728","reported_epoch":"214","state":"active+clean","last_fresh":"2020-11-16T22:35:57.000134+0000","last_change":"2020-11-16T22:35:57.000134+0000","last_active":"2020-11-16T22:35:57.000134+0000","last_peered":"2020-11-16T22:35:57.000134+0000","last_clean":"2020-11-16T22:35:57.000134+0000","last_became_active":"2020-11-16T22:35:56.993064+0000","last_became_peered":"2020-11-16T22:35:56.993064+0000","last_unstale":"2020-11-16T22:35:57.000134+0000","last_undegraded":"2020-11-16T22:35:57.000134+0000","last_fullsized":"2020-11-16T22:35:57.000134+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"151","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984352+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:35:55.984352+0000","last_peered":"2020-11-16T22:35:55.984352+0000","last_clean":"2020-11-16T22:35:55.984352+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:35:55.984352+0000","last_undegraded":"2020-11-16T22:35:55.984352+0000","last_fullsized":"2020-11-16T22:35:55.984352+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4578","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984503+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:35:55.984503+0000","last_peered":"2020-11-16T22:35:55.984503+0000","last_clean":"2020-11-16T22:35:55.984503+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:35:55.984503+0000","last_undegraded":"2020-11-16T22:35:55.984503+0000","last_fullsized":"2020-11-16T22:35:55.984503+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6908","reported_epoch":"216","state":"active+clean","last_fresh":"2020-11-16T22:35:59.054649+0000","last_change":"2020-11-16T22:35:59.054649+0000","last_active":"2020-11-16T22:35:59.054649+0000","last_peered":"2020-11-16T22:35:59.054649+0000","last_clean":"2020-11-16T22:35:59.054649+0000","last_became_active":"2020-11-16T22:35:59.047813+0000","last_became_peered":"2020-11-16T22:35:59.047813+0000","last_unstale":"2020-11-16T22:35:59.054649+0000","last_undegraded":"2020-11-16T22:35:59.054649+0000","last_fullsized":"2020-11-16T22:35:59.054649+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1363,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"90","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041589+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:58.041589+0000","last_peered":"2020-11-16T22:35:58.041589+0000","last_clean":"2020-11-16T22:35:58.041589+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:58.041589+0000","last_undegraded":"2020-11-16T22:35:58.041589+0000","last_fullsized":"2020-11-16T22:35:58.041589+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11054","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984489+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:35:55.984489+0000","last_peered":"2020-11-16T22:35:55.984489+0000","last_clean":"2020-11-16T22:35:55.984489+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:35:55.984489+0000","last_undegraded":"2020-11-16T22:35:55.984489+0000","last_fullsized":"2020-11-16T22:35:55.984489+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23990","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984559+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:55.984559+0000","last_peered":"2020-11-16T22:35:55.984559+0000","last_clean":"2020-11-16T22:35:55.984559+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:55.984559+0000","last_undegraded":"2020-11-16T22:35:55.984559+0000","last_fullsized":"2020-11-16T22:35:55.984559+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"174","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984384+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:35:55.984384+0000","last_peered":"2020-11-16T22:35:55.984384+0000","last_clean":"2020-11-16T22:35:55.984384+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:35:55.984384+0000","last_undegraded":"2020-11-16T22:35:55.984384+0000","last_fullsized":"2020-11-16T22:35:55.984384+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"160","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984221+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:35:55.984221+0000","last_peered":"2020-11-16T22:35:55.984221+0000","last_clean":"2020-11-16T22:35:55.984221+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:35:55.984221+0000","last_undegraded":"2020-11-16T22:35:55.984221+0000","last_fullsized":"2020-11-16T22:35:55.984221+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"8002","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041616+0000","last_change":"2020-11-16T22:35:52.028496+0000","last_active":"2020-11-16T22:35:58.041616+0000","last_peered":"2020-11-16T22:35:58.041616+0000","last_clean":"2020-11-16T22:35:58.041616+0000","last_became_active":"2020-11-16T22:35:52.025147+0000","last_became_peered":"2020-11-16T22:35:52.025147+0000","last_unstale":"2020-11-16T22:35:58.041616+0000","last_undegraded":"2020-11-16T22:35:58.041616+0000","last_fullsized":"2020-11-16T22:35:58.041616+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":39223632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9358","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041877+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:35:58.041877+0000","last_peered":"2020-11-16T22:35:58.041877+0000","last_clean":"2020-11-16T22:35:58.041877+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:35:58.041877+0000","last_undegraded":"2020-11-16T22:35:58.041877+0000","last_fullsized":"2020-11-16T22:35:58.041877+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"156","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984226+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:35:55.984226+0000","last_peered":"2020-11-16T22:35:55.984226+0000","last_clean":"2020-11-16T22:35:55.984226+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:35:55.984226+0000","last_undegraded":"2020-11-16T22:35:55.984226+0000","last_fullsized":"2020-11-16T22:35:55.984226+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"361","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984296+0000","last_change":"2020-11-16T22:35:49.054386+0000","last_active":"2020-11-16T22:35:55.984296+0000","last_peered":"2020-11-16T22:35:55.984296+0000","last_clean":"2020-11-16T22:35:55.984296+0000","last_became_active":"2020-11-16T22:35:48.996346+0000","last_became_peered":"2020-11-16T22:35:48.996346+0000","last_unstale":"2020-11-16T22:35:55.984296+0000","last_undegraded":"2020-11-16T22:35:55.984296+0000","last_fullsized":"2020-11-16T22:35:55.984296+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3949634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33960","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041681+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:58.041681+0000","last_peered":"2020-11-16T22:35:58.041681+0000","last_clean":"2020-11-16T22:35:58.041681+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:58.041681+0000","last_undegraded":"2020-11-16T22:35:58.041681+0000","last_fullsized":"2020-11-16T22:35:58.041681+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"106","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041855+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:35:58.041855+0000","last_peered":"2020-11-16T22:35:58.041855+0000","last_clean":"2020-11-16T22:35:58.041855+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:35:58.041855+0000","last_undegraded":"2020-11-16T22:35:58.041855+0000","last_fullsized":"2020-11-16T22:35:58.041855+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2311","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.983123+0000","last_change":"2020-11-16T22:35:53.024437+0000","last_active":"2020-11-16T22:35:55.983123+0000","last_peered":"2020-11-16T22:35:55.983123+0000","last_clean":"2020-11-16T22:35:55.983123+0000","last_became_active":"2020-11-16T22:35:53.023864+0000","last_became_peered":"2020-11-16T22:35:53.023864+0000","last_unstale":"2020-11-16T22:35:55.983123+0000","last_undegraded":"2020-11-16T22:35:55.983123+0000","last_fullsized":"2020-11-16T22:35:55.983123+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"28461","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984353+0000","last_change":"2020-11-16T22:35:54.995752+0000","last_active":"2020-11-16T22:35:55.984353+0000","last_peered":"2020-11-16T22:35:55.984353+0000","last_clean":"2020-11-16T22:35:55.984353+0000","last_became_active":"2020-11-16T22:35:54.988294+0000","last_became_peered":"2020-11-16T22:35:54.988294+0000","last_unstale":"2020-11-16T22:35:55.984353+0000","last_undegraded":"2020-11-16T22:35:55.984353+0000","last_fullsized":"2020-11-16T22:35:55.984353+0000","mapping_epoch":211,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1873,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2320","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.042533+0000","last_change":"2020-11-16T22:35:52.023474+0000","last_active":"2020-11-16T22:35:58.042533+0000","last_peered":"2020-11-16T22:35:58.042533+0000","last_clean":"2020-11-16T22:35:58.042533+0000","last_became_active":"2020-11-16T22:35:52.022453+0000","last_became_peered":"2020-11-16T22:35:52.022453+0000","last_unstale":"2020-11-16T22:35:58.042533+0000","last_undegraded":"2020-11-16T22:35:58.042533+0000","last_fullsized":"2020-11-16T22:35:58.042533+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"110","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041841+0000","last_change":"2020-11-16T22:35:50.081501+0000","last_active":"2020-11-16T22:35:58.041841+0000","last_peered":"2020-11-16T22:35:58.041841+0000","last_clean":"2020-11-16T22:35:58.041841+0000","last_became_active":"2020-11-16T22:35:50.080825+0000","last_became_peered":"2020-11-16T22:35:50.080825+0000","last_unstale":"2020-11-16T22:35:58.041841+0000","last_undegraded":"2020-11-16T22:35:58.041841+0000","last_fullsized":"2020-11-16T22:35:58.041841+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24216","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041603+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:35:58.041603+0000","last_peered":"2020-11-16T22:35:58.041603+0000","last_clean":"2020-11-16T22:35:58.041603+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:35:58.041603+0000","last_undegraded":"2020-11-16T22:35:58.041603+0000","last_fullsized":"2020-11-16T22:35:58.041603+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"156","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984688+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:35:55.984688+0000","last_peered":"2020-11-16T22:35:55.984688+0000","last_clean":"2020-11-16T22:35:55.984688+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:35:55.984688+0000","last_undegraded":"2020-11-16T22:35:55.984688+0000","last_fullsized":"2020-11-16T22:35:55.984688+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2392","reported_epoch":"213","state":"active+clean","last_fresh":"2020-11-16T22:35:55.984656+0000","last_change":"2020-11-16T22:35:52.021670+0000","last_active":"2020-11-16T22:35:55.984656+0000","last_peered":"2020-11-16T22:35:55.984656+0000","last_clean":"2020-11-16T22:35:55.984656+0000","last_became_active":"2020-11-16T22:35:52.020590+0000","last_became_peered":"2020-11-16T22:35:52.020590+0000","last_unstale":"2020-11-16T22:35:55.984656+0000","last_undegraded":"2020-11-16T22:35:55.984656+0000","last_fullsized":"2020-11-16T22:35:55.984656+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":16777216,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8307,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":143508375,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172294144,"data_stored":259996487,"data_compressed":22249761,"data_compressed_allocated":87773184,"data_compressed_original":175546368,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293788,"kb_used_data":72404,"kb_used_omap":26,"kb_used_meta":221349,"kb_avail":94078052,"statfs":{"total":96636764160,"available":96335925248,"internally_reserved":0,"allocated":74141696,"data_stored":105756666,"data_compressed":8203488,"data_compressed_allocated":32829440,"data_compressed_original":65658880,"omap_allocated":27325,"internal_metadata":226661699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Nov 16 22:35:01 2020","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55800000000000005,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.032,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.008,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.69399999999999995}]}]},{"osd":0,"up_from":205,"seq":880468295683,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79992,"kb_used_data":48816,"kb_used_omap":90,"kb_used_meta":31077,"kb_avail":94291848,"statfs":{"total":96636764160,"available":96554852352,"internally_reserved":0,"allocated":49987584,"data_stored":94347790,"data_compressed":11664311,"data_compressed_allocated":46104576,"data_compressed_original":92209152,"omap_allocated":92596,"internal_metadata":31823436},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":1,"up_from":8,"seq":34359738524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356412,"kb_used_data":54644,"kb_used_omap":0,"kb_used_meta":301760,"kb_avail":94015428,"statfs":{"total":96636764160,"available":96271798272,"internally_reserved":0,"allocated":55955456,"data_stored":63849412,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309002240},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.72799999999999998}]}]}]}} 2020-11-16T22:36:06.840 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:07.139 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:07.140 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:07.151 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2020-11-16T22:36:05.796162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8376,"num_bytes_recovered":176080985,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97234,"ondisk_log_size":97234,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707340,"kb_used_data":151412,"kb_used_omap":117,"kb_used_meta":555786,"kb_avail":282408180,"statfs":{"total":289910292480,"available":289185976320,"internally_reserved":0,"allocated":155045888,"data_stored":229241736,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569125775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001763"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5201","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041604+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:35:58.041604+0000","last_peered":"2020-11-16T22:35:58.041604+0000","last_clean":"2020-11-16T22:35:58.041604+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:35:58.041604+0000","last_undegraded":"2020-11-16T22:35:58.041604+0000","last_fullsized":"2020-11-16T22:35:58.041604+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"154","reported_epoch":"216","state":"active+clean","last_fresh":"2020-11-16T22:35:59.040910+0000","last_change":"2020-11-16T22:36:00.122427+0000","last_active":"2020-11-16T22:35:59.040910+0000","last_peered":"2020-11-16T22:35:59.040910+0000","last_clean":"2020-11-16T22:35:59.040910+0000","last_became_active":"2020-11-16T22:35:48.988556+0000","last_became_peered":"2020-11-16T22:35:48.988556+0000","last_unstale":"2020-11-16T22:35:59.040910+0000","last_undegraded":"2020-11-16T22:35:59.040910+0000","last_fullsized":"2020-11-16T22:35:59.040910+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5588","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041199+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:35:58.041199+0000","last_peered":"2020-11-16T22:35:58.041199+0000","last_clean":"2020-11-16T22:35:58.041199+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:35:58.041199+0000","last_undegraded":"2020-11-16T22:35:58.041199+0000","last_fullsized":"2020-11-16T22:35:58.041199+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17730","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.039813+0000","last_change":"2020-11-16T22:35:57.000134+0000","last_active":"2020-11-16T22:35:58.039813+0000","last_peered":"2020-11-16T22:35:58.039813+0000","last_clean":"2020-11-16T22:35:58.039813+0000","last_became_active":"2020-11-16T22:35:56.993064+0000","last_became_peered":"2020-11-16T22:35:56.993064+0000","last_unstale":"2020-11-16T22:35:58.039813+0000","last_undegraded":"2020-11-16T22:35:58.039813+0000","last_fullsized":"2020-11-16T22:35:58.039813+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"153","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041581+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:35:58.041581+0000","last_peered":"2020-11-16T22:35:58.041581+0000","last_clean":"2020-11-16T22:35:58.041581+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:35:58.041581+0000","last_undegraded":"2020-11-16T22:35:58.041581+0000","last_fullsized":"2020-11-16T22:35:58.041581+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4580","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041088+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:35:58.041088+0000","last_peered":"2020-11-16T22:35:58.041088+0000","last_clean":"2020-11-16T22:35:58.041088+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:35:58.041088+0000","last_undegraded":"2020-11-16T22:35:58.041088+0000","last_fullsized":"2020-11-16T22:35:58.041088+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6908","reported_epoch":"216","state":"active+clean","last_fresh":"2020-11-16T22:35:59.054649+0000","last_change":"2020-11-16T22:35:59.054649+0000","last_active":"2020-11-16T22:35:59.054649+0000","last_peered":"2020-11-16T22:35:59.054649+0000","last_clean":"2020-11-16T22:35:59.054649+0000","last_became_active":"2020-11-16T22:35:59.047813+0000","last_became_peered":"2020-11-16T22:35:59.047813+0000","last_unstale":"2020-11-16T22:35:59.054649+0000","last_undegraded":"2020-11-16T22:35:59.054649+0000","last_fullsized":"2020-11-16T22:35:59.054649+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1363,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"90","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041589+0000","last_change":"2020-11-16T22:35:48.982381+0000","last_active":"2020-11-16T22:35:58.041589+0000","last_peered":"2020-11-16T22:35:58.041589+0000","last_clean":"2020-11-16T22:35:58.041589+0000","last_became_active":"2020-11-16T22:35:48.982100+0000","last_became_peered":"2020-11-16T22:35:48.982100+0000","last_unstale":"2020-11-16T22:35:58.041589+0000","last_undegraded":"2020-11-16T22:35:58.041589+0000","last_fullsized":"2020-11-16T22:35:58.041589+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11056","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041004+0000","last_change":"2020-11-16T22:30:16.104378+0000","last_active":"2020-11-16T22:35:58.041004+0000","last_peered":"2020-11-16T22:35:58.041004+0000","last_clean":"2020-11-16T22:35:58.041004+0000","last_became_active":"2020-11-16T22:30:02.026718+0000","last_became_peered":"2020-11-16T22:30:02.026718+0000","last_unstale":"2020-11-16T22:35:58.041004+0000","last_undegraded":"2020-11-16T22:35:58.041004+0000","last_fullsized":"2020-11-16T22:35:58.041004+0000","mapping_epoch":111,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23992","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041198+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:35:58.041198+0000","last_peered":"2020-11-16T22:35:58.041198+0000","last_clean":"2020-11-16T22:35:58.041198+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:35:58.041198+0000","last_undegraded":"2020-11-16T22:35:58.041198+0000","last_fullsized":"2020-11-16T22:35:58.041198+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"176","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.040982+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:35:58.040982+0000","last_peered":"2020-11-16T22:35:58.040982+0000","last_clean":"2020-11-16T22:35:58.040982+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:35:58.040982+0000","last_undegraded":"2020-11-16T22:35:58.040982+0000","last_fullsized":"2020-11-16T22:35:58.040982+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"162","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.040872+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:35:58.040872+0000","last_peered":"2020-11-16T22:35:58.040872+0000","last_clean":"2020-11-16T22:35:58.040872+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:35:58.040872+0000","last_undegraded":"2020-11-16T22:35:58.040872+0000","last_fullsized":"2020-11-16T22:35:58.040872+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"8002","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041616+0000","last_change":"2020-11-16T22:35:52.028496+0000","last_active":"2020-11-16T22:35:58.041616+0000","last_peered":"2020-11-16T22:35:58.041616+0000","last_clean":"2020-11-16T22:35:58.041616+0000","last_became_active":"2020-11-16T22:35:52.025147+0000","last_became_peered":"2020-11-16T22:35:52.025147+0000","last_unstale":"2020-11-16T22:35:58.041616+0000","last_undegraded":"2020-11-16T22:35:58.041616+0000","last_fullsized":"2020-11-16T22:35:58.041616+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":39223632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9358","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041877+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:35:58.041877+0000","last_peered":"2020-11-16T22:35:58.041877+0000","last_clean":"2020-11-16T22:35:58.041877+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:35:58.041877+0000","last_undegraded":"2020-11-16T22:35:58.041877+0000","last_fullsized":"2020-11-16T22:35:58.041877+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"158","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.040865+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:35:58.040865+0000","last_peered":"2020-11-16T22:35:58.040865+0000","last_clean":"2020-11-16T22:35:58.040865+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:35:58.040865+0000","last_undegraded":"2020-11-16T22:35:58.040865+0000","last_fullsized":"2020-11-16T22:35:58.040865+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"363","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041141+0000","last_change":"2020-11-16T22:35:49.054386+0000","last_active":"2020-11-16T22:35:58.041141+0000","last_peered":"2020-11-16T22:35:58.041141+0000","last_clean":"2020-11-16T22:35:58.041141+0000","last_became_active":"2020-11-16T22:35:48.996346+0000","last_became_peered":"2020-11-16T22:35:48.996346+0000","last_unstale":"2020-11-16T22:35:58.041141+0000","last_undegraded":"2020-11-16T22:35:58.041141+0000","last_fullsized":"2020-11-16T22:35:58.041141+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3949634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33960","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041681+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:35:58.041681+0000","last_peered":"2020-11-16T22:35:58.041681+0000","last_clean":"2020-11-16T22:35:58.041681+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:35:58.041681+0000","last_undegraded":"2020-11-16T22:35:58.041681+0000","last_fullsized":"2020-11-16T22:35:58.041681+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"106","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041855+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:35:58.041855+0000","last_peered":"2020-11-16T22:35:58.041855+0000","last_clean":"2020-11-16T22:35:58.041855+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:35:58.041855+0000","last_undegraded":"2020-11-16T22:35:58.041855+0000","last_fullsized":"2020-11-16T22:35:58.041855+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2313","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.039722+0000","last_change":"2020-11-16T22:35:53.024437+0000","last_active":"2020-11-16T22:35:58.039722+0000","last_peered":"2020-11-16T22:35:58.039722+0000","last_clean":"2020-11-16T22:35:58.039722+0000","last_became_active":"2020-11-16T22:35:53.023864+0000","last_became_peered":"2020-11-16T22:35:53.023864+0000","last_unstale":"2020-11-16T22:35:58.039722+0000","last_undegraded":"2020-11-16T22:35:58.039722+0000","last_fullsized":"2020-11-16T22:35:58.039722+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"28463","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.040995+0000","last_change":"2020-11-16T22:35:54.995752+0000","last_active":"2020-11-16T22:35:58.040995+0000","last_peered":"2020-11-16T22:35:58.040995+0000","last_clean":"2020-11-16T22:35:58.040995+0000","last_became_active":"2020-11-16T22:35:54.988294+0000","last_became_peered":"2020-11-16T22:35:54.988294+0000","last_unstale":"2020-11-16T22:35:58.040995+0000","last_undegraded":"2020-11-16T22:35:58.040995+0000","last_fullsized":"2020-11-16T22:35:58.040995+0000","mapping_epoch":211,"log_start":"140'3300","ondisk_log_start":"140'3300","created":15,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10057,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1873,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2320","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.042533+0000","last_change":"2020-11-16T22:35:52.023474+0000","last_active":"2020-11-16T22:35:58.042533+0000","last_peered":"2020-11-16T22:35:58.042533+0000","last_clean":"2020-11-16T22:35:58.042533+0000","last_became_active":"2020-11-16T22:35:52.022453+0000","last_became_peered":"2020-11-16T22:35:52.022453+0000","last_unstale":"2020-11-16T22:35:58.042533+0000","last_undegraded":"2020-11-16T22:35:58.042533+0000","last_fullsized":"2020-11-16T22:35:58.042533+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"110","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041841+0000","last_change":"2020-11-16T22:35:50.081501+0000","last_active":"2020-11-16T22:35:58.041841+0000","last_peered":"2020-11-16T22:35:58.041841+0000","last_clean":"2020-11-16T22:35:58.041841+0000","last_became_active":"2020-11-16T22:35:50.080825+0000","last_became_peered":"2020-11-16T22:35:50.080825+0000","last_unstale":"2020-11-16T22:35:58.041841+0000","last_undegraded":"2020-11-16T22:35:58.041841+0000","last_fullsized":"2020-11-16T22:35:58.041841+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24216","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041603+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:35:58.041603+0000","last_peered":"2020-11-16T22:35:58.041603+0000","last_clean":"2020-11-16T22:35:58.041603+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:35:58.041603+0000","last_undegraded":"2020-11-16T22:35:58.041603+0000","last_fullsized":"2020-11-16T22:35:58.041603+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"158","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041179+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:35:58.041179+0000","last_peered":"2020-11-16T22:35:58.041179+0000","last_clean":"2020-11-16T22:35:58.041179+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:35:58.041179+0000","last_undegraded":"2020-11-16T22:35:58.041179+0000","last_fullsized":"2020-11-16T22:35:58.041179+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2394","reported_epoch":"215","state":"active+clean","last_fresh":"2020-11-16T22:35:58.041037+0000","last_change":"2020-11-16T22:35:52.021670+0000","last_active":"2020-11-16T22:35:58.041037+0000","last_peered":"2020-11-16T22:35:58.041037+0000","last_clean":"2020-11-16T22:35:58.041037+0000","last_became_active":"2020-11-16T22:35:52.020590+0000","last_became_peered":"2020-11-16T22:35:52.020590+0000","last_unstale":"2020-11-16T22:35:58.041037+0000","last_undegraded":"2020-11-16T22:35:58.041037+0000","last_fullsized":"2020-11-16T22:35:58.041037+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":16777216,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8307,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62816,"ondisk_log_size":62816,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":143508375,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468042,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271992,"kb_used_data":49328,"kb_used_omap":26,"kb_used_meta":222629,"kb_avail":94099848,"statfs":{"total":96636764160,"available":96358244352,"internally_reserved":0,"allocated":50511872,"data_stored":72210082,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":227972419},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.94899999999999995}]}]},{"osd":0,"up_from":205,"seq":880468295684,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78936,"kb_used_data":47440,"kb_used_omap":90,"kb_used_meta":31397,"kb_avail":94292904,"statfs":{"total":96636764160,"available":96555933696,"internally_reserved":0,"allocated":48578560,"data_stored":93182242,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32151116},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356412,"kb_used_data":54644,"kb_used_omap":0,"kb_used_meta":301760,"kb_avail":94015428,"statfs":{"total":96636764160,"available":96271798272,"internally_reserved":0,"allocated":55955456,"data_stored":63849412,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309002240},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Nov 16 22:35:06 2020","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.876,"5min":0.998,"15min":0.998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.3439999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.79700000000000004}]}]}]}} 2020-11-16T22:36:08.579 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-11-16T22:36:08.580 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T22:36:08.580 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-11-16T22:36:08.580 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T22:36:08.580 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T22:36:08.581 INFO:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-11-16T22:36:08.716 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2020-11-16T22:36:08.716 INFO:teuthology.orchestra.run.smithi194.stdout: "success": "" 2020-11-16T22:36:08.716 INFO:teuthology.orchestra.run.smithi194.stdout:} 2020-11-16T22:36:13.725 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2020-11-16T22:36:13.868 INFO:teuthology.orchestra.run.smithi138.stdout:mon.a: {} 2020-11-16T22:36:13.877 INFO:teuthology.orchestra.run.smithi138.stdout:mon.b: {} 2020-11-16T22:36:13.887 INFO:teuthology.orchestra.run.smithi138.stdout:mon.c: {} 2020-11-16T22:36:13.888 INFO:teuthology.orchestra.run.smithi138.stderr:mon.a: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2020-11-16T22:36:13.888 INFO:teuthology.orchestra.run.smithi138.stderr:mon.b: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2020-11-16T22:36:13.889 INFO:teuthology.orchestra.run.smithi138.stderr:mon.c: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2020-11-16T22:36:13.899 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2020-11-16T22:36:15.122 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.1 to 1 (10000) 2020-11-16T22:36:15.135 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2020-11-16T22:36:16.131 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.1 primary-affinity to 1 (8655362) 2020-11-16T22:36:16.142 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2020-11-16T22:36:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.2 to 1 (10000) 2020-11-16T22:36:17.146 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2020-11-16T22:36:18.138 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.2 primary-affinity to 1 (8655362) 2020-11-16T22:36:18.151 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2020-11-16T22:36:19.152 INFO:teuthology.orchestra.run.smithi138.stderr:reweighted osd.0 to 1 (10000) 2020-11-16T22:36:19.164 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2020-11-16T22:36:20.157 INFO:teuthology.orchestra.run.smithi138.stderr:set osd.0 primary-affinity to 1 (8655362) 2020-11-16T22:36:20.169 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2020-11-16T22:36:20.169 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2020-11-16T22:36:20.169 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2020-11-16T22:36:20.170 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2020-11-16T22:36:20.170 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-16T22:36:20.170 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:20.459 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:20.459 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":222,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:36:20.164068+0000","last_up_change":"2020-11-16T22:35:47.976064+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":140,"up_from":205,"up_thru":213,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":6889},{"type":"v1","addr":"172.21.15.173:6801","nonce":6889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":6889},{"type":"v1","addr":"172.21.15.173:6803","nonce":6889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":6889},{"type":"v1","addr":"172.21.15.173:6807","nonce":6889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":6889},{"type":"v1","addr":"172.21.15.173:6805","nonce":6889}]},"public_addr":"172.21.15.173:6801/6889","cluster_addr":"172.21.15.173:6803/6889","heartbeat_back_addr":"172.21.15.173:6807/6889","heartbeat_front_addr":"172.21.15.173:6805/6889","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":63,"last_clean_end":69,"up_from":108,"up_thru":214,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2959},{"type":"v1","addr":"172.21.15.122:6801","nonce":2959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2959},{"type":"v1","addr":"172.21.15.122:6803","nonce":2959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2959},{"type":"v1","addr":"172.21.15.122:6807","nonce":2959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2959},{"type":"v1","addr":"172.21.15.122:6805","nonce":2959}]},"public_addr":"172.21.15.122:6801/2959","cluster_addr":"172.21.15.122:6803/2959","heartbeat_back_addr":"172.21.15.122:6807/2959","heartbeat_front_addr":"172.21.15.122:6805/2959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":70}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:20.472 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-16T22:36:20.472 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:20.760 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":222,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:36:20.164068+0000","last_up_change":"2020-11-16T22:35:47.976064+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":140,"up_from":205,"up_thru":213,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":6889},{"type":"v1","addr":"172.21.15.173:6801","nonce":6889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":6889},{"type":"v1","addr":"172.21.15.173:6803","nonce":6889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":6889},{"type":"v1","addr":"172.21.15.173:6807","nonce":6889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":6889},{"type":"v1","addr":"172.21.15.173:6805","nonce":6889}]},"public_addr":"172.21.15.173:6801/6889","cluster_addr":"172.21.15.173:6803/6889","heartbeat_back_addr":"172.21.15.173:6807/6889","heartbeat_front_addr":"172.21.15.173:6805/6889","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":63,"last_clean_end":69,"up_from":108,"up_thru":214,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2959},{"type":"v1","addr":"172.21.15.122:6801","nonce":2959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2959},{"type":"v1","addr":"172.21.15.122:6803","nonce":2959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2959},{"type":"v1","addr":"172.21.15.122:6807","nonce":2959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2959},{"type":"v1","addr":"172.21.15.122:6805","nonce":2959}]},"public_addr":"172.21.15.122:6801/2959","cluster_addr":"172.21.15.122:6803/2959","heartbeat_back_addr":"172.21.15.122:6807/2959","heartbeat_front_addr":"172.21.15.122:6805/2959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":70}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:20.773 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-16T22:36:20.924 INFO:teuthology.orchestra.run.smithi138.stdout:880468295688 2020-11-16T22:36:20.925 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-16T22:36:21.076 INFO:teuthology.orchestra.run.smithi138.stdout:34359738529 2020-11-16T22:36:21.076 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-16T22:36:21.231 INFO:teuthology.orchestra.run.smithi138.stdout:463856468046 2020-11-16T22:36:21.231 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:36:21.520 INFO:teuthology.orchestra.run.smithi138.stdout:880468295687 2020-11-16T22:36:21.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 880468295688 got 880468295687 for osd.0 2020-11-16T22:36:22.534 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:36:22.831 INFO:teuthology.orchestra.run.smithi138.stdout:880468295688 2020-11-16T22:36:22.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 880468295688 got 880468295688 for osd.0 2020-11-16T22:36:22.844 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-16T22:36:23.130 INFO:teuthology.orchestra.run.smithi138.stdout:34359738529 2020-11-16T22:36:23.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738529 got 34359738529 for osd.1 2020-11-16T22:36:23.143 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:36:23.431 INFO:teuthology.orchestra.run.smithi138.stdout:463856468046 2020-11-16T22:36:23.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856468046 got 463856468046 for osd.2 2020-11-16T22:36:23.444 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-11-16T22:36:23.445 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:23.729 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:23.739 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2020-11-16T22:36:21.799472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100534,"ondisk_log_size":100534,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707668,"kb_used_data":151548,"kb_used_omap":117,"kb_used_meta":555978,"kb_avail":282407852,"statfs":{"total":289910292480,"available":289185640448,"internally_reserved":0,"allocated":155185152,"data_stored":229313755,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569322383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1130,"num_bytes_recovered":-24921154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000807"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5207","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164221+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:36:19.164221+0000","last_peered":"2020-11-16T22:36:19.164221+0000","last_clean":"2020-11-16T22:36:19.164221+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:19.164221+0000","last_undegraded":"2020-11-16T22:36:19.164221+0000","last_fullsized":"2020-11-16T22:36:19.164221+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"157","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168098+0000","last_change":"2020-11-16T22:36:20.168359+0000","last_active":"2020-11-16T22:35:48.974485+0000","last_peered":"2020-11-16T22:34:53.990440+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.113247+0000","last_became_peered":"2020-11-16T22:31:04.113247+0000","last_unstale":"2020-11-16T22:36:20.168098+0000","last_undegraded":"2020-11-16T22:36:20.168098+0000","last_fullsized":"2020-11-16T22:36:20.168098+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5595","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169340+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:36:20.169340+0000","last_peered":"2020-11-16T22:36:20.169340+0000","last_clean":"2020-11-16T22:36:20.169340+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:20.169340+0000","last_undegraded":"2020-11-16T22:36:20.169340+0000","last_fullsized":"2020-11-16T22:36:20.169340+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17733","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.169156+0000","last_change":"2020-11-16T22:36:20.169301+0000","last_active":"2020-11-16T22:36:20.169156+0000","last_peered":"2020-11-16T22:36:15.134567+0000","last_clean":"2020-11-16T22:36:15.134567+0000","last_became_active":"2020-11-16T22:35:56.993064+0000","last_became_peered":"2020-11-16T22:35:56.993064+0000","last_unstale":"2020-11-16T22:36:20.169156+0000","last_undegraded":"2020-11-16T22:36:20.169156+0000","last_fullsized":"2020-11-16T22:36:20.169156+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"160","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169489+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:36:20.169489+0000","last_peered":"2020-11-16T22:36:20.169489+0000","last_clean":"2020-11-16T22:36:20.169489+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:20.169489+0000","last_undegraded":"2020-11-16T22:36:20.169489+0000","last_fullsized":"2020-11-16T22:36:20.169489+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4587","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169505+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:36:20.169505+0000","last_peered":"2020-11-16T22:36:20.169505+0000","last_clean":"2020-11-16T22:36:20.169505+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:20.169505+0000","last_undegraded":"2020-11-16T22:36:20.169505+0000","last_fullsized":"2020-11-16T22:36:20.169505+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6899","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168301+0000","last_change":"2020-11-16T22:36:20.168600+0000","last_active":"2020-11-16T22:35:47.986135+0000","last_peered":"2020-11-16T22:35:00.368843+0000","last_clean":"2020-11-16T22:31:00.092860+0000","last_became_active":"2020-11-16T22:31:04.114321+0000","last_became_peered":"2020-11-16T22:31:04.114321+0000","last_unstale":"2020-11-16T22:36:20.168301+0000","last_undegraded":"2020-11-16T22:36:20.168301+0000","last_fullsized":"2020-11-16T22:36:20.168301+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"133","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168060+0000","last_change":"2020-11-16T22:36:20.168487+0000","last_active":"2020-11-16T22:35:47.984855+0000","last_peered":"2020-11-16T22:34:53.990174+0000","last_clean":"2020-11-16T22:23:13.094572+0000","last_became_active":"2020-11-16T22:31:04.115597+0000","last_became_peered":"2020-11-16T22:31:04.115597+0000","last_unstale":"2020-11-16T22:36:20.168060+0000","last_undegraded":"2020-11-16T22:36:20.168060+0000","last_fullsized":"2020-11-16T22:36:20.168060+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11003","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164033+0000","last_change":"2020-11-16T22:36:17.162490+0000","last_active":"2020-11-16T22:36:19.164033+0000","last_peered":"2020-11-16T22:36:19.164033+0000","last_clean":"2020-11-16T22:36:19.164033+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:19.164033+0000","last_undegraded":"2020-11-16T22:36:19.164033+0000","last_fullsized":"2020-11-16T22:36:19.164033+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23999","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169267+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:36:20.169267+0000","last_peered":"2020-11-16T22:36:20.169267+0000","last_clean":"2020-11-16T22:36:20.169267+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:20.169267+0000","last_undegraded":"2020-11-16T22:36:20.169267+0000","last_fullsized":"2020-11-16T22:36:20.169267+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"183","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169207+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:36:20.169207+0000","last_peered":"2020-11-16T22:36:20.169207+0000","last_clean":"2020-11-16T22:36:20.169207+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:20.169207+0000","last_undegraded":"2020-11-16T22:36:20.169207+0000","last_fullsized":"2020-11-16T22:36:20.169207+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"169","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169068+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:36:20.169068+0000","last_peered":"2020-11-16T22:36:20.169068+0000","last_clean":"2020-11-16T22:36:20.169068+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:20.169068+0000","last_undegraded":"2020-11-16T22:36:20.169068+0000","last_fullsized":"2020-11-16T22:36:20.169068+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7974","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168500+0000","last_change":"2020-11-16T22:36:20.168755+0000","last_active":"2020-11-16T22:35:47.988597+0000","last_peered":"2020-11-16T22:34:59.691417+0000","last_clean":"2020-11-16T22:34:59.691417+0000","last_became_active":"2020-11-16T22:33:23.957142+0000","last_became_peered":"2020-11-16T22:33:23.957142+0000","last_unstale":"2020-11-16T22:36:20.168500+0000","last_undegraded":"2020-11-16T22:36:20.168500+0000","last_fullsized":"2020-11-16T22:36:20.168500+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9364","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164371+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:36:19.164371+0000","last_peered":"2020-11-16T22:36:19.164371+0000","last_clean":"2020-11-16T22:36:19.164371+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:19.164371+0000","last_undegraded":"2020-11-16T22:36:19.164371+0000","last_fullsized":"2020-11-16T22:36:19.164371+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169058+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:36:20.169058+0000","last_peered":"2020-11-16T22:36:20.169058+0000","last_clean":"2020-11-16T22:36:20.169058+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:20.169058+0000","last_undegraded":"2020-11-16T22:36:20.169058+0000","last_fullsized":"2020-11-16T22:36:20.169058+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"342","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.167894+0000","last_change":"2020-11-16T22:36:20.168224+0000","last_active":"2020-11-16T22:35:47.984448+0000","last_peered":"2020-11-16T22:34:59.216900+0000","last_clean":"2020-11-16T22:27:10.078212+0000","last_became_active":"2020-11-16T22:31:04.113920+0000","last_became_peered":"2020-11-16T22:31:04.113920+0000","last_unstale":"2020-11-16T22:36:20.167894+0000","last_undegraded":"2020-11-16T22:36:20.167894+0000","last_fullsized":"2020-11-16T22:36:20.167894+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33966","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164218+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:36:19.164218+0000","last_peered":"2020-11-16T22:36:19.164218+0000","last_clean":"2020-11-16T22:36:19.164218+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:19.164218+0000","last_undegraded":"2020-11-16T22:36:19.164218+0000","last_fullsized":"2020-11-16T22:36:19.164218+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"112","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164241+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:36:19.164241+0000","last_peered":"2020-11-16T22:36:19.164241+0000","last_clean":"2020-11-16T22:36:19.164241+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:19.164241+0000","last_undegraded":"2020-11-16T22:36:19.164241+0000","last_fullsized":"2020-11-16T22:36:19.164241+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2316","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168317+0000","last_change":"2020-11-16T22:36:20.168592+0000","last_active":"2020-11-16T22:36:20.168317+0000","last_peered":"2020-11-16T22:36:15.134447+0000","last_clean":"2020-11-16T22:36:15.134447+0000","last_became_active":"2020-11-16T22:35:53.023864+0000","last_became_peered":"2020-11-16T22:35:53.023864+0000","last_unstale":"2020-11-16T22:36:20.168317+0000","last_undegraded":"2020-11-16T22:36:20.168317+0000","last_fullsized":"2020-11-16T22:36:20.168317+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27563","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.167841+0000","last_change":"2020-11-16T22:36:20.168194+0000","last_active":"2020-11-16T22:35:47.986402+0000","last_peered":"2020-11-16T22:35:00.349182+0000","last_clean":"2020-11-16T22:35:00.349182+0000","last_became_active":"2020-11-16T22:33:23.953893+0000","last_became_peered":"2020-11-16T22:33:23.953893+0000","last_unstale":"2020-11-16T22:36:20.167841+0000","last_undegraded":"2020-11-16T22:36:20.167841+0000","last_fullsized":"2020-11-16T22:36:20.167841+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2296","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.169031+0000","last_change":"2020-11-16T22:36:20.169185+0000","last_active":"2020-11-16T22:35:47.987103+0000","last_peered":"2020-11-16T22:34:59.330740+0000","last_clean":"2020-11-16T22:29:54.995405+0000","last_became_active":"2020-11-16T22:31:04.115488+0000","last_became_peered":"2020-11-16T22:31:04.115488+0000","last_unstale":"2020-11-16T22:36:20.169031+0000","last_undegraded":"2020-11-16T22:36:20.169031+0000","last_fullsized":"2020-11-16T22:36:20.169031+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"96","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168332+0000","last_change":"2020-11-16T22:36:20.168596+0000","last_active":"2020-11-16T22:32:00.114224+0000","last_peered":"2020-11-16T22:31:56.100192+0000","last_clean":"2020-11-16T22:31:56.100192+0000","last_became_active":"2020-11-16T22:31:25.006214+0000","last_became_peered":"2020-11-16T22:31:25.006214+0000","last_unstale":"2020-11-16T22:36:20.168332+0000","last_undegraded":"2020-11-16T22:36:20.168332+0000","last_fullsized":"2020-11-16T22:36:20.168332+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24222","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164166+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:36:19.164166+0000","last_peered":"2020-11-16T22:36:19.164166+0000","last_clean":"2020-11-16T22:36:19.164166+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:19.164166+0000","last_undegraded":"2020-11-16T22:36:19.164166+0000","last_fullsized":"2020-11-16T22:36:19.164166+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169300+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:36:20.169300+0000","last_peered":"2020-11-16T22:36:20.169300+0000","last_clean":"2020-11-16T22:36:20.169300+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:20.169300+0000","last_undegraded":"2020-11-16T22:36:20.169300+0000","last_fullsized":"2020-11-16T22:36:20.169300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2368","reported_epoch":"222","state":"peering","last_fresh":"2020-11-16T22:36:20.168343+0000","last_change":"2020-11-16T22:36:20.168604+0000","last_active":"2020-11-16T22:35:47.988141+0000","last_peered":"2020-11-16T22:34:58.902027+0000","last_clean":"2020-11-16T22:34:58.902027+0000","last_became_active":"2020-11-16T22:32:02.132057+0000","last_became_peered":"2020-11-16T22:32:02.132057+0000","last_unstale":"2020-11-16T22:36:20.168343+0000","last_undegraded":"2020-11-16T22:36:20.168343+0000","last_fullsized":"2020-11-16T22:36:20.168343+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468046,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272104,"kb_used_data":49376,"kb_used_omap":26,"kb_used_meta":222693,"kb_avail":94099736,"statfs":{"total":96636764160,"available":96358129664,"internally_reserved":0,"allocated":50561024,"data_stored":72235500,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228037955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.92300000000000004}]}]},{"osd":0,"up_from":205,"seq":880468295688,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79048,"kb_used_data":47488,"kb_used_omap":90,"kb_used_meta":31461,"kb_avail":94292792,"statfs":{"total":96636764160,"available":96555819008,"internally_reserved":0,"allocated":48627712,"data_stored":93207660,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32216652},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356516,"kb_used_data":54684,"kb_used_omap":0,"kb_used_meta":301824,"kb_avail":94015324,"statfs":{"total":96636764160,"available":96271691776,"internally_reserved":0,"allocated":55996416,"data_stored":63870595,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309067776},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.95899999999999996}]},{"osd":2,"last update":"Mon Nov 16 22:36:08 2020","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.56999999999999995,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.056},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.624,"15min":0.56699999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1659999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.98899999999999999}]}]}]}} 2020-11-16T22:36:24.147 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:24.454 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":617,"stamp":"2020-11-16T22:36:23.799857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100534,"ondisk_log_size":100534,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707812,"kb_used_data":151564,"kb_used_omap":117,"kb_used_meta":556106,"kb_avail":282407708,"statfs":{"total":289910292480,"available":289185492992,"internally_reserved":0,"allocated":155201536,"data_stored":229322225,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569453455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1130,"num_bytes_recovered":-24921154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.651068"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5207","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164221+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:36:19.164221+0000","last_peered":"2020-11-16T22:36:19.164221+0000","last_clean":"2020-11-16T22:36:19.164221+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:19.164221+0000","last_undegraded":"2020-11-16T22:36:19.164221+0000","last_fullsized":"2020-11-16T22:36:19.164221+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"165","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178257+0000","last_change":"2020-11-16T22:36:21.178257+0000","last_active":"2020-11-16T22:36:21.178257+0000","last_peered":"2020-11-16T22:36:21.178257+0000","last_clean":"2020-11-16T22:36:21.178257+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:21.178257+0000","last_undegraded":"2020-11-16T22:36:21.178257+0000","last_fullsized":"2020-11-16T22:36:21.178257+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5595","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169340+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:36:20.169340+0000","last_peered":"2020-11-16T22:36:20.169340+0000","last_clean":"2020-11-16T22:36:20.169340+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:20.169340+0000","last_undegraded":"2020-11-16T22:36:20.169340+0000","last_fullsized":"2020-11-16T22:36:20.169340+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17741","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.193781+0000","last_change":"2020-11-16T22:36:21.193781+0000","last_active":"2020-11-16T22:36:21.193781+0000","last_peered":"2020-11-16T22:36:21.193781+0000","last_clean":"2020-11-16T22:36:21.193781+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:36:21.193781+0000","last_undegraded":"2020-11-16T22:36:21.193781+0000","last_fullsized":"2020-11-16T22:36:21.193781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"160","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169489+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:36:20.169489+0000","last_peered":"2020-11-16T22:36:20.169489+0000","last_clean":"2020-11-16T22:36:20.169489+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:20.169489+0000","last_undegraded":"2020-11-16T22:36:20.169489+0000","last_fullsized":"2020-11-16T22:36:20.169489+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4587","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169505+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:36:20.169505+0000","last_peered":"2020-11-16T22:36:20.169505+0000","last_clean":"2020-11-16T22:36:20.169505+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:20.169505+0000","last_undegraded":"2020-11-16T22:36:20.169505+0000","last_fullsized":"2020-11-16T22:36:20.169505+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6907","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192270+0000","last_change":"2020-11-16T22:36:21.192270+0000","last_active":"2020-11-16T22:36:21.192270+0000","last_peered":"2020-11-16T22:36:21.192270+0000","last_clean":"2020-11-16T22:36:21.192270+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:36:21.192270+0000","last_undegraded":"2020-11-16T22:36:21.192270+0000","last_fullsized":"2020-11-16T22:36:21.192270+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"141","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.174623+0000","last_change":"2020-11-16T22:36:21.174623+0000","last_active":"2020-11-16T22:36:21.174623+0000","last_peered":"2020-11-16T22:36:21.174623+0000","last_clean":"2020-11-16T22:36:21.174623+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:21.174623+0000","last_undegraded":"2020-11-16T22:36:21.174623+0000","last_fullsized":"2020-11-16T22:36:21.174623+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11003","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164033+0000","last_change":"2020-11-16T22:36:17.162490+0000","last_active":"2020-11-16T22:36:19.164033+0000","last_peered":"2020-11-16T22:36:19.164033+0000","last_clean":"2020-11-16T22:36:19.164033+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:19.164033+0000","last_undegraded":"2020-11-16T22:36:19.164033+0000","last_fullsized":"2020-11-16T22:36:19.164033+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23999","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169267+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:36:20.169267+0000","last_peered":"2020-11-16T22:36:20.169267+0000","last_clean":"2020-11-16T22:36:20.169267+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:20.169267+0000","last_undegraded":"2020-11-16T22:36:20.169267+0000","last_fullsized":"2020-11-16T22:36:20.169267+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"183","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169207+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:36:20.169207+0000","last_peered":"2020-11-16T22:36:20.169207+0000","last_clean":"2020-11-16T22:36:20.169207+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:20.169207+0000","last_undegraded":"2020-11-16T22:36:20.169207+0000","last_fullsized":"2020-11-16T22:36:20.169207+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"169","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169068+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:36:20.169068+0000","last_peered":"2020-11-16T22:36:20.169068+0000","last_clean":"2020-11-16T22:36:20.169068+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:20.169068+0000","last_undegraded":"2020-11-16T22:36:20.169068+0000","last_fullsized":"2020-11-16T22:36:20.169068+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7982","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.179105+0000","last_change":"2020-11-16T22:36:21.179105+0000","last_active":"2020-11-16T22:36:21.179105+0000","last_peered":"2020-11-16T22:36:21.179105+0000","last_clean":"2020-11-16T22:36:21.179105+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:21.179105+0000","last_undegraded":"2020-11-16T22:36:21.179105+0000","last_fullsized":"2020-11-16T22:36:21.179105+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9364","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164371+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:36:19.164371+0000","last_peered":"2020-11-16T22:36:19.164371+0000","last_clean":"2020-11-16T22:36:19.164371+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:19.164371+0000","last_undegraded":"2020-11-16T22:36:19.164371+0000","last_fullsized":"2020-11-16T22:36:19.164371+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169058+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:36:20.169058+0000","last_peered":"2020-11-16T22:36:20.169058+0000","last_clean":"2020-11-16T22:36:20.169058+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:20.169058+0000","last_undegraded":"2020-11-16T22:36:20.169058+0000","last_fullsized":"2020-11-16T22:36:20.169058+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"350","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.173590+0000","last_change":"2020-11-16T22:36:21.173590+0000","last_active":"2020-11-16T22:36:21.173590+0000","last_peered":"2020-11-16T22:36:21.173590+0000","last_clean":"2020-11-16T22:36:21.173590+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:21.173590+0000","last_undegraded":"2020-11-16T22:36:21.173590+0000","last_fullsized":"2020-11-16T22:36:21.173590+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33966","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164218+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:36:19.164218+0000","last_peered":"2020-11-16T22:36:19.164218+0000","last_clean":"2020-11-16T22:36:19.164218+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:19.164218+0000","last_undegraded":"2020-11-16T22:36:19.164218+0000","last_fullsized":"2020-11-16T22:36:19.164218+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"112","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164241+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:36:19.164241+0000","last_peered":"2020-11-16T22:36:19.164241+0000","last_clean":"2020-11-16T22:36:19.164241+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:19.164241+0000","last_undegraded":"2020-11-16T22:36:19.164241+0000","last_fullsized":"2020-11-16T22:36:19.164241+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2324","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178619+0000","last_change":"2020-11-16T22:36:21.178619+0000","last_active":"2020-11-16T22:36:21.178619+0000","last_peered":"2020-11-16T22:36:21.178619+0000","last_clean":"2020-11-16T22:36:21.178619+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:21.178619+0000","last_undegraded":"2020-11-16T22:36:21.178619+0000","last_fullsized":"2020-11-16T22:36:21.178619+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27571","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192149+0000","last_change":"2020-11-16T22:36:21.192149+0000","last_active":"2020-11-16T22:36:21.192149+0000","last_peered":"2020-11-16T22:36:21.192149+0000","last_clean":"2020-11-16T22:36:21.192149+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:36:21.192149+0000","last_undegraded":"2020-11-16T22:36:21.192149+0000","last_fullsized":"2020-11-16T22:36:21.192149+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2304","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178871+0000","last_change":"2020-11-16T22:36:21.178871+0000","last_active":"2020-11-16T22:36:21.178871+0000","last_peered":"2020-11-16T22:36:21.178871+0000","last_clean":"2020-11-16T22:36:21.178871+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:36:21.178871+0000","last_undegraded":"2020-11-16T22:36:21.178871+0000","last_fullsized":"2020-11-16T22:36:21.178871+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"104","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.177647+0000","last_change":"2020-11-16T22:36:21.177647+0000","last_active":"2020-11-16T22:36:21.177647+0000","last_peered":"2020-11-16T22:36:21.177647+0000","last_clean":"2020-11-16T22:36:21.177647+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:21.177647+0000","last_undegraded":"2020-11-16T22:36:21.177647+0000","last_fullsized":"2020-11-16T22:36:21.177647+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24222","reported_epoch":"221","state":"active+clean","last_fresh":"2020-11-16T22:36:19.164166+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:36:19.164166+0000","last_peered":"2020-11-16T22:36:19.164166+0000","last_clean":"2020-11-16T22:36:19.164166+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:19.164166+0000","last_undegraded":"2020-11-16T22:36:19.164166+0000","last_fullsized":"2020-11-16T22:36:19.164166+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169300+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:36:20.169300+0000","last_peered":"2020-11-16T22:36:20.169300+0000","last_clean":"2020-11-16T22:36:20.169300+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:20.169300+0000","last_undegraded":"2020-11-16T22:36:20.169300+0000","last_fullsized":"2020-11-16T22:36:20.169300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2376","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.176728+0000","last_change":"2020-11-16T22:36:21.176728+0000","last_active":"2020-11-16T22:36:21.176728+0000","last_peered":"2020-11-16T22:36:21.176728+0000","last_clean":"2020-11-16T22:36:21.176728+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:36:21.176728+0000","last_undegraded":"2020-11-16T22:36:21.176728+0000","last_fullsized":"2020-11-16T22:36:21.176728+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468047,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.177},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":1.131}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.337}]}]},{"osd":0,"up_from":205,"seq":880468295689,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79120,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31525,"kb_avail":94292720,"statfs":{"total":96636764160,"available":96555745280,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32282188},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356516,"kb_used_data":54684,"kb_used_omap":0,"kb_used_meta":301824,"kb_avail":94015324,"statfs":{"total":96636764160,"available":96271691776,"internally_reserved":0,"allocated":55996416,"data_stored":63870595,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309067776},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.95899999999999996}]},{"osd":2,"last update":"Mon Nov 16 22:36:08 2020","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.56999999999999995,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.056},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.624,"15min":0.56699999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1659999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.98899999999999999}]}]}]}} 2020-11-16T22:36:25.271 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-11-16T22:36:25.272 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-11-16T22:36:25.590 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-11-16T22:36:25.600 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-11-16T22:36:25.600 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:25.892 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:25.893 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":223,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:36:21.165000+0000","last_up_change":"2020-11-16T22:35:47.976064+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":140,"up_from":205,"up_thru":222,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":6889},{"type":"v1","addr":"172.21.15.173:6801","nonce":6889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":6889},{"type":"v1","addr":"172.21.15.173:6803","nonce":6889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":6889},{"type":"v1","addr":"172.21.15.173:6807","nonce":6889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":6889},{"type":"v1","addr":"172.21.15.173:6805","nonce":6889}]},"public_addr":"172.21.15.173:6801/6889","cluster_addr":"172.21.15.173:6803/6889","heartbeat_back_addr":"172.21.15.173:6807/6889","heartbeat_front_addr":"172.21.15.173:6805/6889","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":63,"last_clean_end":69,"up_from":108,"up_thru":214,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2959},{"type":"v1","addr":"172.21.15.122:6801","nonce":2959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2959},{"type":"v1","addr":"172.21.15.122:6803","nonce":2959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2959},{"type":"v1","addr":"172.21.15.122:6807","nonce":2959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2959},{"type":"v1","addr":"172.21.15.122:6805","nonce":2959}]},"public_addr":"172.21.15.122:6801/2959","cluster_addr":"172.21.15.122:6803/2959","heartbeat_back_addr":"172.21.15.122:6807/2959","heartbeat_front_addr":"172.21.15.122:6805/2959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":70}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:25.905 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:26.194 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:26.204 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":618,"stamp":"2020-11-16T22:36:25.800200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100534,"ondisk_log_size":100534,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707892,"kb_used_data":151580,"kb_used_omap":117,"kb_used_meta":556170,"kb_avail":282407628,"statfs":{"total":289910292480,"available":289185411072,"internally_reserved":0,"allocated":155217920,"data_stored":229330695,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569518991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1130,"num_bytes_recovered":-24921154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.639359"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5208","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170556+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:36:20.170556+0000","last_peered":"2020-11-16T22:36:20.170556+0000","last_clean":"2020-11-16T22:36:20.170556+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:20.170556+0000","last_undegraded":"2020-11-16T22:36:20.170556+0000","last_fullsized":"2020-11-16T22:36:20.170556+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"165","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178257+0000","last_change":"2020-11-16T22:36:21.178257+0000","last_active":"2020-11-16T22:36:21.178257+0000","last_peered":"2020-11-16T22:36:21.178257+0000","last_clean":"2020-11-16T22:36:21.178257+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:21.178257+0000","last_undegraded":"2020-11-16T22:36:21.178257+0000","last_fullsized":"2020-11-16T22:36:21.178257+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5595","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169340+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:36:20.169340+0000","last_peered":"2020-11-16T22:36:20.169340+0000","last_clean":"2020-11-16T22:36:20.169340+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:20.169340+0000","last_undegraded":"2020-11-16T22:36:20.169340+0000","last_fullsized":"2020-11-16T22:36:20.169340+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17741","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.193781+0000","last_change":"2020-11-16T22:36:21.193781+0000","last_active":"2020-11-16T22:36:21.193781+0000","last_peered":"2020-11-16T22:36:21.193781+0000","last_clean":"2020-11-16T22:36:21.193781+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:36:21.193781+0000","last_undegraded":"2020-11-16T22:36:21.193781+0000","last_fullsized":"2020-11-16T22:36:21.193781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"160","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169489+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:36:20.169489+0000","last_peered":"2020-11-16T22:36:20.169489+0000","last_clean":"2020-11-16T22:36:20.169489+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:20.169489+0000","last_undegraded":"2020-11-16T22:36:20.169489+0000","last_fullsized":"2020-11-16T22:36:20.169489+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4587","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169505+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:36:20.169505+0000","last_peered":"2020-11-16T22:36:20.169505+0000","last_clean":"2020-11-16T22:36:20.169505+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:20.169505+0000","last_undegraded":"2020-11-16T22:36:20.169505+0000","last_fullsized":"2020-11-16T22:36:20.169505+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6907","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192270+0000","last_change":"2020-11-16T22:36:21.192270+0000","last_active":"2020-11-16T22:36:21.192270+0000","last_peered":"2020-11-16T22:36:21.192270+0000","last_clean":"2020-11-16T22:36:21.192270+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:36:21.192270+0000","last_undegraded":"2020-11-16T22:36:21.192270+0000","last_fullsized":"2020-11-16T22:36:21.192270+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"141","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.174623+0000","last_change":"2020-11-16T22:36:21.174623+0000","last_active":"2020-11-16T22:36:21.174623+0000","last_peered":"2020-11-16T22:36:21.174623+0000","last_clean":"2020-11-16T22:36:21.174623+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:21.174623+0000","last_undegraded":"2020-11-16T22:36:21.174623+0000","last_fullsized":"2020-11-16T22:36:21.174623+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11004","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169605+0000","last_change":"2020-11-16T22:36:17.162490+0000","last_active":"2020-11-16T22:36:20.169605+0000","last_peered":"2020-11-16T22:36:20.169605+0000","last_clean":"2020-11-16T22:36:20.169605+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:20.169605+0000","last_undegraded":"2020-11-16T22:36:20.169605+0000","last_fullsized":"2020-11-16T22:36:20.169605+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23999","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169267+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:36:20.169267+0000","last_peered":"2020-11-16T22:36:20.169267+0000","last_clean":"2020-11-16T22:36:20.169267+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:20.169267+0000","last_undegraded":"2020-11-16T22:36:20.169267+0000","last_fullsized":"2020-11-16T22:36:20.169267+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"183","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169207+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:36:20.169207+0000","last_peered":"2020-11-16T22:36:20.169207+0000","last_clean":"2020-11-16T22:36:20.169207+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:20.169207+0000","last_undegraded":"2020-11-16T22:36:20.169207+0000","last_fullsized":"2020-11-16T22:36:20.169207+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"169","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169068+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:36:20.169068+0000","last_peered":"2020-11-16T22:36:20.169068+0000","last_clean":"2020-11-16T22:36:20.169068+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:20.169068+0000","last_undegraded":"2020-11-16T22:36:20.169068+0000","last_fullsized":"2020-11-16T22:36:20.169068+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7982","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.179105+0000","last_change":"2020-11-16T22:36:21.179105+0000","last_active":"2020-11-16T22:36:21.179105+0000","last_peered":"2020-11-16T22:36:21.179105+0000","last_clean":"2020-11-16T22:36:21.179105+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:21.179105+0000","last_undegraded":"2020-11-16T22:36:21.179105+0000","last_fullsized":"2020-11-16T22:36:21.179105+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9365","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170279+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:36:20.170279+0000","last_peered":"2020-11-16T22:36:20.170279+0000","last_clean":"2020-11-16T22:36:20.170279+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:20.170279+0000","last_undegraded":"2020-11-16T22:36:20.170279+0000","last_fullsized":"2020-11-16T22:36:20.170279+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169058+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:36:20.169058+0000","last_peered":"2020-11-16T22:36:20.169058+0000","last_clean":"2020-11-16T22:36:20.169058+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:20.169058+0000","last_undegraded":"2020-11-16T22:36:20.169058+0000","last_fullsized":"2020-11-16T22:36:20.169058+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"350","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.173590+0000","last_change":"2020-11-16T22:36:21.173590+0000","last_active":"2020-11-16T22:36:21.173590+0000","last_peered":"2020-11-16T22:36:21.173590+0000","last_clean":"2020-11-16T22:36:21.173590+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:21.173590+0000","last_undegraded":"2020-11-16T22:36:21.173590+0000","last_fullsized":"2020-11-16T22:36:21.173590+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33967","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170128+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:36:20.170128+0000","last_peered":"2020-11-16T22:36:20.170128+0000","last_clean":"2020-11-16T22:36:20.170128+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:20.170128+0000","last_undegraded":"2020-11-16T22:36:20.170128+0000","last_fullsized":"2020-11-16T22:36:20.170128+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"113","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170229+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:36:20.170229+0000","last_peered":"2020-11-16T22:36:20.170229+0000","last_clean":"2020-11-16T22:36:20.170229+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:20.170229+0000","last_undegraded":"2020-11-16T22:36:20.170229+0000","last_fullsized":"2020-11-16T22:36:20.170229+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2324","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178619+0000","last_change":"2020-11-16T22:36:21.178619+0000","last_active":"2020-11-16T22:36:21.178619+0000","last_peered":"2020-11-16T22:36:21.178619+0000","last_clean":"2020-11-16T22:36:21.178619+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:21.178619+0000","last_undegraded":"2020-11-16T22:36:21.178619+0000","last_fullsized":"2020-11-16T22:36:21.178619+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27571","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192149+0000","last_change":"2020-11-16T22:36:21.192149+0000","last_active":"2020-11-16T22:36:21.192149+0000","last_peered":"2020-11-16T22:36:21.192149+0000","last_clean":"2020-11-16T22:36:21.192149+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:36:21.192149+0000","last_undegraded":"2020-11-16T22:36:21.192149+0000","last_fullsized":"2020-11-16T22:36:21.192149+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2304","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178871+0000","last_change":"2020-11-16T22:36:21.178871+0000","last_active":"2020-11-16T22:36:21.178871+0000","last_peered":"2020-11-16T22:36:21.178871+0000","last_clean":"2020-11-16T22:36:21.178871+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:36:21.178871+0000","last_undegraded":"2020-11-16T22:36:21.178871+0000","last_fullsized":"2020-11-16T22:36:21.178871+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"104","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.177647+0000","last_change":"2020-11-16T22:36:21.177647+0000","last_active":"2020-11-16T22:36:21.177647+0000","last_peered":"2020-11-16T22:36:21.177647+0000","last_clean":"2020-11-16T22:36:21.177647+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:21.177647+0000","last_undegraded":"2020-11-16T22:36:21.177647+0000","last_fullsized":"2020-11-16T22:36:21.177647+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24223","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169468+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:36:20.169468+0000","last_peered":"2020-11-16T22:36:20.169468+0000","last_clean":"2020-11-16T22:36:20.169468+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:20.169468+0000","last_undegraded":"2020-11-16T22:36:20.169468+0000","last_fullsized":"2020-11-16T22:36:20.169468+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169300+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:36:20.169300+0000","last_peered":"2020-11-16T22:36:20.169300+0000","last_clean":"2020-11-16T22:36:20.169300+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:20.169300+0000","last_undegraded":"2020-11-16T22:36:20.169300+0000","last_fullsized":"2020-11-16T22:36:20.169300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2376","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.176728+0000","last_change":"2020-11-16T22:36:21.176728+0000","last_active":"2020-11-16T22:36:21.176728+0000","last_peered":"2020-11-16T22:36:21.176728+0000","last_clean":"2020-11-16T22:36:21.176728+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:36:21.176728+0000","last_undegraded":"2020-11-16T22:36:21.176728+0000","last_fullsized":"2020-11-16T22:36:21.176728+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468047,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.177},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":1.131}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.337}]}]},{"osd":0,"up_from":205,"seq":880468295689,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79120,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31525,"kb_avail":94292720,"statfs":{"total":96636764160,"available":96555745280,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32282188},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356596,"kb_used_data":54700,"kb_used_omap":0,"kb_used_meta":301888,"kb_avail":94015244,"statfs":{"total":96636764160,"available":96271609856,"internally_reserved":0,"allocated":56012800,"data_stored":63879065,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309133312},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.628}]},{"osd":2,"last update":"Mon Nov 16 22:36:08 2020","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.56999999999999995,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.624,"15min":0.56699999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1659999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.65500000000000003}]}]}]}} 2020-11-16T22:36:26.407 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:36:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:26.699 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":223,"fsid":"2963e9ac-8003-490b-a7b3-aee3b9109ae1","created":"2020-11-16T22:23:04.049027+0000","modified":"2020-11-16T22:36:21.165000+0000","last_up_change":"2020-11-16T22:35:47.976064+0000","last_in_change":"2020-11-16T22:23:09.075128+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:23:07.667948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:23:12.436040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:23:15.658351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:23:16.461191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d7ac8050-ab58-4ba1-8b67-54b1751e09c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":140,"up_from":205,"up_thru":222,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":6889},{"type":"v1","addr":"172.21.15.173:6801","nonce":6889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":6889},{"type":"v1","addr":"172.21.15.173:6803","nonce":6889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":6889},{"type":"v1","addr":"172.21.15.173:6807","nonce":6889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":6889},{"type":"v1","addr":"172.21.15.173:6805","nonce":6889}]},"public_addr":"172.21.15.173:6801/6889","cluster_addr":"172.21.15.173:6803/6889","heartbeat_back_addr":"172.21.15.173:6807/6889","heartbeat_front_addr":"172.21.15.173:6805/6889","state":["exists","up"]},{"osd":1,"uuid":"94f7af5a-9a19-4ee0-8078-95c6902e33f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":20794},{"type":"v1","addr":"172.21.15.194:6801","nonce":20794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":20794},{"type":"v1","addr":"172.21.15.194:6803","nonce":20794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":20794},{"type":"v1","addr":"172.21.15.194:6807","nonce":20794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":20794},{"type":"v1","addr":"172.21.15.194:6805","nonce":20794}]},"public_addr":"172.21.15.194:6801/20794","cluster_addr":"172.21.15.194:6803/20794","heartbeat_back_addr":"172.21.15.194:6807/20794","heartbeat_front_addr":"172.21.15.194:6805/20794","state":["exists","up"]},{"osd":2,"uuid":"92637938-4ff7-497e-b599-40612e2d47dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":63,"last_clean_end":69,"up_from":108,"up_thru":214,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2959},{"type":"v1","addr":"172.21.15.122:6801","nonce":2959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2959},{"type":"v1","addr":"172.21.15.122:6803","nonce":2959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2959},{"type":"v1","addr":"172.21.15.122:6807","nonce":2959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2959},{"type":"v1","addr":"172.21.15.122:6805","nonce":2959}]},"public_addr":"172.21.15.122:6801/2959","cluster_addr":"172.21.15.122:6803/2959","heartbeat_back_addr":"172.21.15.122:6807/2959","heartbeat_front_addr":"172.21.15.122:6805/2959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.886258+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.939013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:23:07.955568+0000","dead_epoch":70}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:36:27.712 INFO:tasks.ceph:Scrubbing osd.0 2020-11-16T22:36:27.712 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:27.868 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:36:27.869 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:27.869 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:36:27.879 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2020-11-16T22:36:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 0 to deep-scrub 2020-11-16T22:36:28.183 INFO:tasks.ceph:Scrubbing osd.1 2020-11-16T22:36:28.183 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:28.325 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:36:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:28.326 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:36:28.337 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2020-11-16T22:36:28.627 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 1 to deep-scrub 2020-11-16T22:36:28.637 INFO:tasks.ceph:Scrubbing osd.2 2020-11-16T22:36:28.638 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-11-16T22:36:28.780 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2020-11-16T22:36:28.780 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-16T22:36:28.780 INFO:teuthology.orchestra.run.smithi138.stdout:} 2020-11-16T22:36:28.791 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2020-11-16T22:36:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 2 to deep-scrub 2020-11-16T22:36:29.094 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:29.381 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:29.382 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:29.393 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":619,"stamp":"2020-11-16T22:36:27.800547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100534,"ondisk_log_size":100534,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707892,"kb_used_data":151580,"kb_used_omap":117,"kb_used_meta":556170,"kb_avail":282407628,"statfs":{"total":289910292480,"available":289185411072,"internally_reserved":0,"allocated":155217920,"data_stored":229330695,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569518991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1130,"num_bytes_recovered":-24921154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001413"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5208","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170556+0000","last_change":"2020-11-16T22:35:51.021873+0000","last_active":"2020-11-16T22:36:20.170556+0000","last_peered":"2020-11-16T22:36:20.170556+0000","last_clean":"2020-11-16T22:36:20.170556+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:20.170556+0000","last_undegraded":"2020-11-16T22:36:20.170556+0000","last_fullsized":"2020-11-16T22:36:20.170556+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"165","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178257+0000","last_change":"2020-11-16T22:36:21.178257+0000","last_active":"2020-11-16T22:36:21.178257+0000","last_peered":"2020-11-16T22:36:21.178257+0000","last_clean":"2020-11-16T22:36:21.178257+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:21.178257+0000","last_undegraded":"2020-11-16T22:36:21.178257+0000","last_fullsized":"2020-11-16T22:36:21.178257+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5595","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169340+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:36:20.169340+0000","last_peered":"2020-11-16T22:36:20.169340+0000","last_clean":"2020-11-16T22:36:20.169340+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:20.169340+0000","last_undegraded":"2020-11-16T22:36:20.169340+0000","last_fullsized":"2020-11-16T22:36:20.169340+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17741","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.193781+0000","last_change":"2020-11-16T22:36:21.193781+0000","last_active":"2020-11-16T22:36:21.193781+0000","last_peered":"2020-11-16T22:36:21.193781+0000","last_clean":"2020-11-16T22:36:21.193781+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:36:21.193781+0000","last_undegraded":"2020-11-16T22:36:21.193781+0000","last_fullsized":"2020-11-16T22:36:21.193781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"160","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169489+0000","last_change":"2020-11-16T22:35:48.983586+0000","last_active":"2020-11-16T22:36:20.169489+0000","last_peered":"2020-11-16T22:36:20.169489+0000","last_clean":"2020-11-16T22:36:20.169489+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:20.169489+0000","last_undegraded":"2020-11-16T22:36:20.169489+0000","last_fullsized":"2020-11-16T22:36:20.169489+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"204'4372","reported_seq":"4587","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169505+0000","last_change":"2020-11-16T22:35:49.099877+0000","last_active":"2020-11-16T22:36:20.169505+0000","last_peered":"2020-11-16T22:36:20.169505+0000","last_clean":"2020-11-16T22:36:20.169505+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:20.169505+0000","last_undegraded":"2020-11-16T22:36:20.169505+0000","last_fullsized":"2020-11-16T22:36:20.169505+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":4372,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6907","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192270+0000","last_change":"2020-11-16T22:36:21.192270+0000","last_active":"2020-11-16T22:36:21.192270+0000","last_peered":"2020-11-16T22:36:21.192270+0000","last_clean":"2020-11-16T22:36:21.192270+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:36:21.192270+0000","last_undegraded":"2020-11-16T22:36:21.192270+0000","last_fullsized":"2020-11-16T22:36:21.192270+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"141","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.174623+0000","last_change":"2020-11-16T22:36:21.174623+0000","last_active":"2020-11-16T22:36:21.174623+0000","last_peered":"2020-11-16T22:36:21.174623+0000","last_clean":"2020-11-16T22:36:21.174623+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:21.174623+0000","last_undegraded":"2020-11-16T22:36:21.174623+0000","last_fullsized":"2020-11-16T22:36:21.174623+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"204'10826","reported_seq":"11004","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169605+0000","last_change":"2020-11-16T22:36:17.162490+0000","last_active":"2020-11-16T22:36:20.169605+0000","last_peered":"2020-11-16T22:36:20.169605+0000","last_clean":"2020-11-16T22:36:20.169605+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:20.169605+0000","last_undegraded":"2020-11-16T22:36:20.169605+0000","last_fullsized":"2020-11-16T22:36:20.169605+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":10026,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23999","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169267+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:36:20.169267+0000","last_peered":"2020-11-16T22:36:20.169267+0000","last_clean":"2020-11-16T22:36:20.169267+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:20.169267+0000","last_undegraded":"2020-11-16T22:36:20.169267+0000","last_fullsized":"2020-11-16T22:36:20.169267+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"183","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169207+0000","last_change":"2020-11-16T22:35:50.000523+0000","last_active":"2020-11-16T22:36:20.169207+0000","last_peered":"2020-11-16T22:36:20.169207+0000","last_clean":"2020-11-16T22:36:20.169207+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:20.169207+0000","last_undegraded":"2020-11-16T22:36:20.169207+0000","last_fullsized":"2020-11-16T22:36:20.169207+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'4","reported_seq":"169","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169068+0000","last_change":"2020-11-16T22:35:50.032160+0000","last_active":"2020-11-16T22:36:20.169068+0000","last_peered":"2020-11-16T22:36:20.169068+0000","last_clean":"2020-11-16T22:36:20.169068+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:20.169068+0000","last_undegraded":"2020-11-16T22:36:20.169068+0000","last_fullsized":"2020-11-16T22:36:20.169068+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:08.065273+0000","last_clean_scrub_stamp":"2020-11-16T22:23:08.065273+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7982","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.179105+0000","last_change":"2020-11-16T22:36:21.179105+0000","last_active":"2020-11-16T22:36:21.179105+0000","last_peered":"2020-11-16T22:36:21.179105+0000","last_clean":"2020-11-16T22:36:21.179105+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:21.179105+0000","last_undegraded":"2020-11-16T22:36:21.179105+0000","last_fullsized":"2020-11-16T22:36:21.179105+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9365","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170279+0000","last_change":"2020-11-16T22:35:55.998207+0000","last_active":"2020-11-16T22:36:20.170279+0000","last_peered":"2020-11-16T22:36:20.170279+0000","last_clean":"2020-11-16T22:36:20.170279+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:20.170279+0000","last_undegraded":"2020-11-16T22:36:20.170279+0000","last_fullsized":"2020-11-16T22:36:20.170279+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169058+0000","last_change":"2020-11-16T22:35:50.032077+0000","last_active":"2020-11-16T22:36:20.169058+0000","last_peered":"2020-11-16T22:36:20.169058+0000","last_clean":"2020-11-16T22:36:20.169058+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:20.169058+0000","last_undegraded":"2020-11-16T22:36:20.169058+0000","last_fullsized":"2020-11-16T22:36:20.169058+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"350","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.173590+0000","last_change":"2020-11-16T22:36:21.173590+0000","last_active":"2020-11-16T22:36:21.173590+0000","last_peered":"2020-11-16T22:36:21.173590+0000","last_clean":"2020-11-16T22:36:21.173590+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:21.173590+0000","last_undegraded":"2020-11-16T22:36:21.173590+0000","last_fullsized":"2020-11-16T22:36:21.173590+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33967","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170128+0000","last_change":"2020-11-16T22:30:51.070745+0000","last_active":"2020-11-16T22:36:20.170128+0000","last_peered":"2020-11-16T22:36:20.170128+0000","last_clean":"2020-11-16T22:36:20.170128+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:20.170128+0000","last_undegraded":"2020-11-16T22:36:20.170128+0000","last_fullsized":"2020-11-16T22:36:20.170128+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"113","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.170229+0000","last_change":"2020-11-16T22:35:49.994832+0000","last_active":"2020-11-16T22:36:20.170229+0000","last_peered":"2020-11-16T22:36:20.170229+0000","last_clean":"2020-11-16T22:36:20.170229+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:20.170229+0000","last_undegraded":"2020-11-16T22:36:20.170229+0000","last_fullsized":"2020-11-16T22:36:20.170229+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2324","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178619+0000","last_change":"2020-11-16T22:36:21.178619+0000","last_active":"2020-11-16T22:36:21.178619+0000","last_peered":"2020-11-16T22:36:21.178619+0000","last_clean":"2020-11-16T22:36:21.178619+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:21.178619+0000","last_undegraded":"2020-11-16T22:36:21.178619+0000","last_fullsized":"2020-11-16T22:36:21.178619+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27571","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192149+0000","last_change":"2020-11-16T22:36:21.192149+0000","last_active":"2020-11-16T22:36:21.192149+0000","last_peered":"2020-11-16T22:36:21.192149+0000","last_clean":"2020-11-16T22:36:21.192149+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:36:21.192149+0000","last_undegraded":"2020-11-16T22:36:21.192149+0000","last_fullsized":"2020-11-16T22:36:21.192149+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2304","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178871+0000","last_change":"2020-11-16T22:36:21.178871+0000","last_active":"2020-11-16T22:36:21.178871+0000","last_peered":"2020-11-16T22:36:21.178871+0000","last_clean":"2020-11-16T22:36:21.178871+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:36:21.178871+0000","last_undegraded":"2020-11-16T22:36:21.178871+0000","last_fullsized":"2020-11-16T22:36:21.178871+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"104","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.177647+0000","last_change":"2020-11-16T22:36:21.177647+0000","last_active":"2020-11-16T22:36:21.177647+0000","last_peered":"2020-11-16T22:36:21.177647+0000","last_clean":"2020-11-16T22:36:21.177647+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:21.177647+0000","last_undegraded":"2020-11-16T22:36:21.177647+0000","last_fullsized":"2020-11-16T22:36:21.177647+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24223","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169468+0000","last_change":"2020-11-16T22:35:51.020286+0000","last_active":"2020-11-16T22:36:20.169468+0000","last_peered":"2020-11-16T22:36:20.169468+0000","last_clean":"2020-11-16T22:36:20.169468+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:20.169468+0000","last_undegraded":"2020-11-16T22:36:20.169468+0000","last_fullsized":"2020-11-16T22:36:20.169468+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"165","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169300+0000","last_change":"2020-11-16T22:35:50.117197+0000","last_active":"2020-11-16T22:36:20.169300+0000","last_peered":"2020-11-16T22:36:20.169300+0000","last_clean":"2020-11-16T22:36:20.169300+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:20.169300+0000","last_undegraded":"2020-11-16T22:36:20.169300+0000","last_fullsized":"2020-11-16T22:36:20.169300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:13.094572+0000","last_clean_scrub_stamp":"2020-11-16T22:23:13.094572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2376","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.176728+0000","last_change":"2020-11-16T22:36:21.176728+0000","last_active":"2020-11-16T22:36:21.176728+0000","last_peered":"2020-11-16T22:36:21.176728+0000","last_clean":"2020-11-16T22:36:21.176728+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:36:21.176728+0000","last_undegraded":"2020-11-16T22:36:21.176728+0000","last_fullsized":"2020-11-16T22:36:21.176728+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34414,"ondisk_log_size":34414,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468048,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.177},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":1.131}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.337}]}]},{"osd":0,"up_from":205,"seq":880468295690,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79120,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31525,"kb_avail":94292720,"statfs":{"total":96636764160,"available":96555745280,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32282188},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356596,"kb_used_data":54700,"kb_used_omap":0,"kb_used_meta":301888,"kb_avail":94015244,"statfs":{"total":96636764160,"available":96271609856,"internally_reserved":0,"allocated":56012800,"data_stored":63879065,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309133312},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:27 2020","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.621,"15min":0.66800000000000004},"min":{"1min":0.35099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84299999999999997,"5min":1.252,"15min":1.252},"last":0.628}]},{"osd":2,"last update":"Mon Nov 16 22:36:08 2020","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.56999999999999995,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.624,"15min":0.56699999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1659999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.65500000000000003}]}]}]}} 2020-11-16T22:36:30.209 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.210 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.210 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.210 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.210 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.211 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.211 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.211 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.211 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.212 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.414 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.414 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-11-16T22:23:08.065273+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=8, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.415 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.415 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.415 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.415 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.416 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.416 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.416 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.416 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.416 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.417 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.417 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.417 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-11-16T22:23:13.094572+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=13, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.417 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:30.418 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-16T22:36:50.418 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:36:50.716 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:36:50.717 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:36:50.728 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":630,"stamp":"2020-11-16T22:36:49.805365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21239,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100555,"ondisk_log_size":100555,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707892,"kb_used_data":151580,"kb_used_omap":117,"kb_used_meta":556170,"kb_avail":282407628,"statfs":{"total":289910292480,"available":289185411072,"internally_reserved":0,"allocated":155217920,"data_stored":229330695,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569518991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002640"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5211","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:38.222678+0000","last_change":"2020-11-16T22:36:38.222761+0000","last_active":"2020-11-16T22:36:38.222678+0000","last_peered":"2020-11-16T22:36:38.222678+0000","last_clean":"2020-11-16T22:36:38.222678+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:38.222678+0000","last_undegraded":"2020-11-16T22:36:38.222678+0000","last_fullsized":"2020-11-16T22:36:38.222678+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'3435","last_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_deep_scrub":"204'3435","last_deep_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_clean_scrub_stamp":"2020-11-16T22:36:38.222672+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:36.268887+0000","last_change":"2020-11-16T22:36:36.268958+0000","last_active":"2020-11-16T22:36:36.268887+0000","last_peered":"2020-11-16T22:36:36.268887+0000","last_clean":"2020-11-16T22:36:36.268887+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:36.268887+0000","last_undegraded":"2020-11-16T22:36:36.268887+0000","last_fullsized":"2020-11-16T22:36:36.268887+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_clean_scrub_stamp":"2020-11-16T22:36:36.268882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"204'5371","reported_seq":"5595","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169340+0000","last_change":"2020-11-16T22:35:52.022222+0000","last_active":"2020-11-16T22:36:20.169340+0000","last_peered":"2020-11-16T22:36:20.169340+0000","last_clean":"2020-11-16T22:36:20.169340+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:20.169340+0000","last_undegraded":"2020-11-16T22:36:20.169340+0000","last_fullsized":"2020-11-16T22:36:20.169340+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":5371,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17741","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.193781+0000","last_change":"2020-11-16T22:36:21.193781+0000","last_active":"2020-11-16T22:36:21.193781+0000","last_peered":"2020-11-16T22:36:21.193781+0000","last_clean":"2020-11-16T22:36:21.193781+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:36:21.193781+0000","last_undegraded":"2020-11-16T22:36:21.193781+0000","last_fullsized":"2020-11-16T22:36:21.193781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"163","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:34.023339+0000","last_change":"2020-11-16T22:36:34.023420+0000","last_active":"2020-11-16T22:36:34.023339+0000","last_peered":"2020-11-16T22:36:34.023339+0000","last_clean":"2020-11-16T22:36:34.023339+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:34.023339+0000","last_undegraded":"2020-11-16T22:36:34.023339+0000","last_fullsized":"2020-11-16T22:36:34.023339+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_clean_scrub_stamp":"2020-11-16T22:36:34.023333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"223'4380","reported_seq":"4598","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:46.029417+0000","last_change":"2020-11-16T22:36:46.029476+0000","last_active":"2020-11-16T22:36:46.029417+0000","last_peered":"2020-11-16T22:36:46.029417+0000","last_clean":"2020-11-16T22:36:46.029417+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:46.029417+0000","last_undegraded":"2020-11-16T22:36:46.029417+0000","last_fullsized":"2020-11-16T22:36:46.029417+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'4380","last_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_deep_scrub":"223'4380","last_deep_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_clean_scrub_stamp":"2020-11-16T22:36:46.029411+0000","log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6907","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192270+0000","last_change":"2020-11-16T22:36:21.192270+0000","last_active":"2020-11-16T22:36:21.192270+0000","last_peered":"2020-11-16T22:36:21.192270+0000","last_clean":"2020-11-16T22:36:21.192270+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:36:21.192270+0000","last_undegraded":"2020-11-16T22:36:21.192270+0000","last_fullsized":"2020-11-16T22:36:21.192270+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"144","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:35.229186+0000","last_change":"2020-11-16T22:36:35.229272+0000","last_active":"2020-11-16T22:36:35.229186+0000","last_peered":"2020-11-16T22:36:35.229186+0000","last_clean":"2020-11-16T22:36:35.229186+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:35.229186+0000","last_undegraded":"2020-11-16T22:36:35.229186+0000","last_fullsized":"2020-11-16T22:36:35.229186+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_clean_scrub_stamp":"2020-11-16T22:36:35.229180+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"223'10835","reported_seq":"11016","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:30.347689+0000","last_change":"2020-11-16T22:36:30.347766+0000","last_active":"2020-11-16T22:36:30.347689+0000","last_peered":"2020-11-16T22:36:30.347689+0000","last_clean":"2020-11-16T22:36:30.347689+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:30.347689+0000","last_undegraded":"2020-11-16T22:36:30.347689+0000","last_fullsized":"2020-11-16T22:36:30.347689+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'10835","last_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_deep_scrub":"223'10835","last_deep_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_clean_scrub_stamp":"2020-11-16T22:36:30.347682+0000","log_size":10035,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"23999","reported_epoch":"222","state":"active+clean","last_fresh":"2020-11-16T22:36:20.169267+0000","last_change":"2020-11-16T22:30:16.020688+0000","last_active":"2020-11-16T22:36:20.169267+0000","last_peered":"2020-11-16T22:36:20.169267+0000","last_clean":"2020-11-16T22:36:20.169267+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:20.169267+0000","last_undegraded":"2020-11-16T22:36:20.169267+0000","last_fullsized":"2020-11-16T22:36:20.169267+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"186","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:37.968557+0000","last_change":"2020-11-16T22:36:37.968642+0000","last_active":"2020-11-16T22:36:37.968557+0000","last_peered":"2020-11-16T22:36:37.968557+0000","last_clean":"2020-11-16T22:36:37.968557+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:37.968557+0000","last_undegraded":"2020-11-16T22:36:37.968557+0000","last_fullsized":"2020-11-16T22:36:37.968557+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_clean_scrub_stamp":"2020-11-16T22:36:37.968551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"223'8","reported_seq":"176","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:32.029740+0000","last_change":"2020-11-16T22:36:32.029809+0000","last_active":"2020-11-16T22:36:32.029740+0000","last_peered":"2020-11-16T22:36:32.029740+0000","last_clean":"2020-11-16T22:36:32.029740+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:32.029740+0000","last_undegraded":"2020-11-16T22:36:32.029740+0000","last_fullsized":"2020-11-16T22:36:32.029740+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'8","last_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_deep_scrub":"223'8","last_deep_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_clean_scrub_stamp":"2020-11-16T22:36:32.029734+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"204'7791","reported_seq":"7982","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.179105+0000","last_change":"2020-11-16T22:36:21.179105+0000","last_active":"2020-11-16T22:36:21.179105+0000","last_peered":"2020-11-16T22:36:21.179105+0000","last_clean":"2020-11-16T22:36:21.179105+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:21.179105+0000","last_undegraded":"2020-11-16T22:36:21.179105+0000","last_fullsized":"2020-11-16T22:36:21.179105+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":7791,"ondisk_log_size":7791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9368","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:41.287297+0000","last_change":"2020-11-16T22:36:41.287378+0000","last_active":"2020-11-16T22:36:41.287297+0000","last_peered":"2020-11-16T22:36:41.287297+0000","last_clean":"2020-11-16T22:36:41.287297+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:41.287297+0000","last_undegraded":"2020-11-16T22:36:41.287297+0000","last_fullsized":"2020-11-16T22:36:41.287297+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'4789","last_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_deep_scrub":"204'4789","last_deep_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_clean_scrub_stamp":"2020-11-16T22:36:41.287291+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.056454+0000","last_change":"2020-11-16T22:36:33.056522+0000","last_active":"2020-11-16T22:36:33.056454+0000","last_peered":"2020-11-16T22:36:33.056454+0000","last_clean":"2020-11-16T22:36:33.056454+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:33.056454+0000","last_undegraded":"2020-11-16T22:36:33.056454+0000","last_fullsized":"2020-11-16T22:36:33.056454+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.056448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"204'117","reported_seq":"350","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.173590+0000","last_change":"2020-11-16T22:36:21.173590+0000","last_active":"2020-11-16T22:36:21.173590+0000","last_peered":"2020-11-16T22:36:21.173590+0000","last_clean":"2020-11-16T22:36:21.173590+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:21.173590+0000","last_undegraded":"2020-11-16T22:36:21.173590+0000","last_fullsized":"2020-11-16T22:36:21.173590+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33970","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:40.276217+0000","last_change":"2020-11-16T22:36:40.276311+0000","last_active":"2020-11-16T22:36:40.276217+0000","last_peered":"2020-11-16T22:36:40.276217+0000","last_clean":"2020-11-16T22:36:40.276217+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:40.276217+0000","last_undegraded":"2020-11-16T22:36:40.276217+0000","last_fullsized":"2020-11-16T22:36:40.276217+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'16676","last_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_deep_scrub":"204'16676","last_deep_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_clean_scrub_stamp":"2020-11-16T22:36:40.276200+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"116","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:29.201097+0000","last_change":"2020-11-16T22:36:29.201173+0000","last_active":"2020-11-16T22:36:29.201097+0000","last_peered":"2020-11-16T22:36:29.201097+0000","last_clean":"2020-11-16T22:36:29.201097+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:29.201097+0000","last_undegraded":"2020-11-16T22:36:29.201097+0000","last_fullsized":"2020-11-16T22:36:29.201097+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_clean_scrub_stamp":"2020-11-16T22:36:29.201091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"204'2269","reported_seq":"2324","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178619+0000","last_change":"2020-11-16T22:36:21.178619+0000","last_active":"2020-11-16T22:36:21.178619+0000","last_peered":"2020-11-16T22:36:21.178619+0000","last_clean":"2020-11-16T22:36:21.178619+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:21.178619+0000","last_undegraded":"2020-11-16T22:36:21.178619+0000","last_fullsized":"2020-11-16T22:36:21.178619+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2269,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27571","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192149+0000","last_change":"2020-11-16T22:36:21.192149+0000","last_active":"2020-11-16T22:36:21.192149+0000","last_peered":"2020-11-16T22:36:21.192149+0000","last_clean":"2020-11-16T22:36:21.192149+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:36:21.192149+0000","last_undegraded":"2020-11-16T22:36:21.192149+0000","last_fullsized":"2020-11-16T22:36:21.192149+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"204'2178","reported_seq":"2304","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.178871+0000","last_change":"2020-11-16T22:36:21.178871+0000","last_active":"2020-11-16T22:36:21.178871+0000","last_peered":"2020-11-16T22:36:21.178871+0000","last_clean":"2020-11-16T22:36:21.178871+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:36:21.178871+0000","last_undegraded":"2020-11-16T22:36:21.178871+0000","last_fullsized":"2020-11-16T22:36:21.178871+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2178,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"107","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:39.333478+0000","last_change":"2020-11-16T22:36:39.333552+0000","last_active":"2020-11-16T22:36:39.333478+0000","last_peered":"2020-11-16T22:36:39.333478+0000","last_clean":"2020-11-16T22:36:39.333478+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:39.333478+0000","last_undegraded":"2020-11-16T22:36:39.333478+0000","last_fullsized":"2020-11-16T22:36:39.333478+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_clean_scrub_stamp":"2020-11-16T22:36:39.333472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24226","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.269459+0000","last_change":"2020-11-16T22:36:33.269536+0000","last_active":"2020-11-16T22:36:33.269459+0000","last_peered":"2020-11-16T22:36:33.269459+0000","last_clean":"2020-11-16T22:36:33.269459+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:33.269459+0000","last_undegraded":"2020-11-16T22:36:33.269459+0000","last_fullsized":"2020-11-16T22:36:33.269459+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'11769","last_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_deep_scrub":"204'11769","last_deep_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.269452+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:44.000800+0000","last_change":"2020-11-16T22:36:44.000867+0000","last_active":"2020-11-16T22:36:44.000800+0000","last_peered":"2020-11-16T22:36:44.000800+0000","last_clean":"2020-11-16T22:36:44.000800+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:44.000800+0000","last_undegraded":"2020-11-16T22:36:44.000800+0000","last_fullsized":"2020-11-16T22:36:44.000800+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_clean_scrub_stamp":"2020-11-16T22:36:44.000794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"204'2290","reported_seq":"2376","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.176728+0000","last_change":"2020-11-16T22:36:21.176728+0000","last_active":"2020-11-16T22:36:21.176728+0000","last_peered":"2020-11-16T22:36:21.176728+0000","last_clean":"2020-11-16T22:36:21.176728+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:36:21.176728+0000","last_undegraded":"2020-11-16T22:36:21.176728+0000","last_fullsized":"2020-11-16T22:36:21.176728+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:16.110463+0000","last_clean_scrub_stamp":"2020-11-16T22:23:16.110463+0000","log_size":2290,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34431,"ondisk_log_size":34431,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468052,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:30:59 2020","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48999999999999999}]},{"osd":1,"last update":"Mon Nov 16 22:36:01 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.58499999999999996,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.004,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.58799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.627}]}]},{"osd":0,"up_from":205,"seq":880468295694,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79120,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31525,"kb_avail":94292720,"statfs":{"total":96636764160,"available":96555745280,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32282188},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356596,"kb_used_data":54700,"kb_used_omap":0,"kb_used_meta":301888,"kb_avail":94015244,"statfs":{"total":96636764160,"available":96271609856,"internally_reserved":0,"allocated":56012800,"data_stored":63879065,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309133312},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:49 2020","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Nov 16 22:36:08 2020","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.56999999999999995,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.624,"15min":0.56699999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1659999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.72699999999999998}]}]}]}} 2020-11-16T22:36:50.730 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.730 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.730 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.731 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.731 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.731 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.731 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.732 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.732 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.732 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2020-11-16T22:23:16.110463+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=16, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:36:50.732 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-16T22:37:10.734 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:37:11.031 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:37:11.032 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:37:11.043 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":640,"stamp":"2020-11-16T22:37:09.810142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21239,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100587,"ondisk_log_size":100587,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707956,"kb_used_data":151580,"kb_used_omap":117,"kb_used_meta":556234,"kb_avail":282407564,"statfs":{"total":289910292480,"available":289185345536,"internally_reserved":0,"allocated":155217920,"data_stored":229330695,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569584527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002841"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5211","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:38.222678+0000","last_change":"2020-11-16T22:36:38.222761+0000","last_active":"2020-11-16T22:36:38.222678+0000","last_peered":"2020-11-16T22:36:38.222678+0000","last_clean":"2020-11-16T22:36:38.222678+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:38.222678+0000","last_undegraded":"2020-11-16T22:36:38.222678+0000","last_fullsized":"2020-11-16T22:36:38.222678+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'3435","last_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_deep_scrub":"204'3435","last_deep_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_clean_scrub_stamp":"2020-11-16T22:36:38.222672+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:36.268887+0000","last_change":"2020-11-16T22:36:36.268958+0000","last_active":"2020-11-16T22:36:36.268887+0000","last_peered":"2020-11-16T22:36:36.268887+0000","last_clean":"2020-11-16T22:36:36.268887+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:36.268887+0000","last_undegraded":"2020-11-16T22:36:36.268887+0000","last_fullsized":"2020-11-16T22:36:36.268887+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_clean_scrub_stamp":"2020-11-16T22:36:36.268882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"223'5379","reported_seq":"5606","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:51.204392+0000","last_change":"2020-11-16T22:36:51.204461+0000","last_active":"2020-11-16T22:36:51.204392+0000","last_peered":"2020-11-16T22:36:51.204392+0000","last_clean":"2020-11-16T22:36:51.204392+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:51.204392+0000","last_undegraded":"2020-11-16T22:36:51.204392+0000","last_fullsized":"2020-11-16T22:36:51.204392+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'5379","last_scrub_stamp":"2020-11-16T22:36:51.204386+0000","last_deep_scrub":"223'5379","last_deep_scrub_stamp":"2020-11-16T22:36:51.204386+0000","last_clean_scrub_stamp":"2020-11-16T22:36:51.204386+0000","log_size":5379,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17744","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:07.053747+0000","last_change":"2020-11-16T22:37:07.053819+0000","last_active":"2020-11-16T22:37:07.053747+0000","last_peered":"2020-11-16T22:37:07.053747+0000","last_clean":"2020-11-16T22:37:07.053747+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:37:07.053747+0000","last_undegraded":"2020-11-16T22:37:07.053747+0000","last_fullsized":"2020-11-16T22:37:07.053747+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'9703","last_scrub_stamp":"2020-11-16T22:37:07.053743+0000","last_deep_scrub":"204'9703","last_deep_scrub_stamp":"2020-11-16T22:37:07.053743+0000","last_clean_scrub_stamp":"2020-11-16T22:37:07.053743+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"163","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:34.023339+0000","last_change":"2020-11-16T22:36:34.023420+0000","last_active":"2020-11-16T22:36:34.023339+0000","last_peered":"2020-11-16T22:36:34.023339+0000","last_clean":"2020-11-16T22:36:34.023339+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:34.023339+0000","last_undegraded":"2020-11-16T22:36:34.023339+0000","last_fullsized":"2020-11-16T22:36:34.023339+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_clean_scrub_stamp":"2020-11-16T22:36:34.023333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"223'4380","reported_seq":"4598","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:46.029417+0000","last_change":"2020-11-16T22:36:46.029476+0000","last_active":"2020-11-16T22:36:46.029417+0000","last_peered":"2020-11-16T22:36:46.029417+0000","last_clean":"2020-11-16T22:36:46.029417+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:46.029417+0000","last_undegraded":"2020-11-16T22:36:46.029417+0000","last_fullsized":"2020-11-16T22:36:46.029417+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'4380","last_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_deep_scrub":"223'4380","last_deep_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_clean_scrub_stamp":"2020-11-16T22:36:46.029411+0000","log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6907","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:21.192270+0000","last_change":"2020-11-16T22:36:21.192270+0000","last_active":"2020-11-16T22:36:21.192270+0000","last_peered":"2020-11-16T22:36:21.192270+0000","last_clean":"2020-11-16T22:36:21.192270+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:36:21.192270+0000","last_undegraded":"2020-11-16T22:36:21.192270+0000","last_fullsized":"2020-11-16T22:36:21.192270+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:23:17.119717+0000","last_clean_scrub_stamp":"2020-11-16T22:23:17.119717+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"144","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:35.229186+0000","last_change":"2020-11-16T22:36:35.229272+0000","last_active":"2020-11-16T22:36:35.229186+0000","last_peered":"2020-11-16T22:36:35.229186+0000","last_clean":"2020-11-16T22:36:35.229186+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:35.229186+0000","last_undegraded":"2020-11-16T22:36:35.229186+0000","last_fullsized":"2020-11-16T22:36:35.229186+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_clean_scrub_stamp":"2020-11-16T22:36:35.229180+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"223'10835","reported_seq":"11016","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:30.347689+0000","last_change":"2020-11-16T22:36:30.347766+0000","last_active":"2020-11-16T22:36:30.347689+0000","last_peered":"2020-11-16T22:36:30.347689+0000","last_clean":"2020-11-16T22:36:30.347689+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:30.347689+0000","last_undegraded":"2020-11-16T22:36:30.347689+0000","last_fullsized":"2020-11-16T22:36:30.347689+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'10835","last_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_deep_scrub":"223'10835","last_deep_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_clean_scrub_stamp":"2020-11-16T22:36:30.347682+0000","log_size":10035,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"24002","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:52.129093+0000","last_change":"2020-11-16T22:36:52.129165+0000","last_active":"2020-11-16T22:36:52.129093+0000","last_peered":"2020-11-16T22:36:52.129093+0000","last_clean":"2020-11-16T22:36:52.129093+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:52.129093+0000","last_undegraded":"2020-11-16T22:36:52.129093+0000","last_fullsized":"2020-11-16T22:36:52.129093+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'17258","last_scrub_stamp":"2020-11-16T22:36:52.129087+0000","last_deep_scrub":"204'17258","last_deep_scrub_stamp":"2020-11-16T22:36:52.129087+0000","last_clean_scrub_stamp":"2020-11-16T22:36:52.129087+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"186","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:37.968557+0000","last_change":"2020-11-16T22:36:37.968642+0000","last_active":"2020-11-16T22:36:37.968557+0000","last_peered":"2020-11-16T22:36:37.968557+0000","last_clean":"2020-11-16T22:36:37.968557+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:37.968557+0000","last_undegraded":"2020-11-16T22:36:37.968557+0000","last_fullsized":"2020-11-16T22:36:37.968557+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_clean_scrub_stamp":"2020-11-16T22:36:37.968551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"223'8","reported_seq":"176","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:32.029740+0000","last_change":"2020-11-16T22:36:32.029809+0000","last_active":"2020-11-16T22:36:32.029740+0000","last_peered":"2020-11-16T22:36:32.029740+0000","last_clean":"2020-11-16T22:36:32.029740+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:32.029740+0000","last_undegraded":"2020-11-16T22:36:32.029740+0000","last_fullsized":"2020-11-16T22:36:32.029740+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'8","last_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_deep_scrub":"223'8","last_deep_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_clean_scrub_stamp":"2020-11-16T22:36:32.029734+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"223'7796","reported_seq":"7990","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:50.343781+0000","last_change":"2020-11-16T22:36:50.343844+0000","last_active":"2020-11-16T22:36:50.343781+0000","last_peered":"2020-11-16T22:36:50.343781+0000","last_clean":"2020-11-16T22:36:50.343781+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:50.343781+0000","last_undegraded":"2020-11-16T22:36:50.343781+0000","last_fullsized":"2020-11-16T22:36:50.343781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'7796","last_scrub_stamp":"2020-11-16T22:36:50.343775+0000","last_deep_scrub":"223'7796","last_deep_scrub_stamp":"2020-11-16T22:36:50.343775+0000","last_clean_scrub_stamp":"2020-11-16T22:36:50.343775+0000","log_size":7796,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9368","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:41.287297+0000","last_change":"2020-11-16T22:36:41.287378+0000","last_active":"2020-11-16T22:36:41.287297+0000","last_peered":"2020-11-16T22:36:41.287297+0000","last_clean":"2020-11-16T22:36:41.287297+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:41.287297+0000","last_undegraded":"2020-11-16T22:36:41.287297+0000","last_fullsized":"2020-11-16T22:36:41.287297+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'4789","last_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_deep_scrub":"204'4789","last_deep_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_clean_scrub_stamp":"2020-11-16T22:36:41.287291+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.056454+0000","last_change":"2020-11-16T22:36:33.056522+0000","last_active":"2020-11-16T22:36:33.056454+0000","last_peered":"2020-11-16T22:36:33.056454+0000","last_clean":"2020-11-16T22:36:33.056454+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:33.056454+0000","last_undegraded":"2020-11-16T22:36:33.056454+0000","last_fullsized":"2020-11-16T22:36:33.056454+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.056448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"223'119","reported_seq":"355","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:48.259053+0000","last_change":"2020-11-16T22:36:48.259135+0000","last_active":"2020-11-16T22:36:48.259053+0000","last_peered":"2020-11-16T22:36:48.259053+0000","last_clean":"2020-11-16T22:36:48.259053+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:48.259053+0000","last_undegraded":"2020-11-16T22:36:48.259053+0000","last_fullsized":"2020-11-16T22:36:48.259053+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'119","last_scrub_stamp":"2020-11-16T22:36:48.259047+0000","last_deep_scrub":"223'119","last_deep_scrub_stamp":"2020-11-16T22:36:48.259047+0000","last_clean_scrub_stamp":"2020-11-16T22:36:48.259047+0000","log_size":119,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33970","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:40.276217+0000","last_change":"2020-11-16T22:36:40.276311+0000","last_active":"2020-11-16T22:36:40.276217+0000","last_peered":"2020-11-16T22:36:40.276217+0000","last_clean":"2020-11-16T22:36:40.276217+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:40.276217+0000","last_undegraded":"2020-11-16T22:36:40.276217+0000","last_fullsized":"2020-11-16T22:36:40.276217+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'16676","last_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_deep_scrub":"204'16676","last_deep_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_clean_scrub_stamp":"2020-11-16T22:36:40.276200+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"116","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:29.201097+0000","last_change":"2020-11-16T22:36:29.201173+0000","last_active":"2020-11-16T22:36:29.201097+0000","last_peered":"2020-11-16T22:36:29.201097+0000","last_clean":"2020-11-16T22:36:29.201097+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:29.201097+0000","last_undegraded":"2020-11-16T22:36:29.201097+0000","last_fullsized":"2020-11-16T22:36:29.201097+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_clean_scrub_stamp":"2020-11-16T22:36:29.201091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"223'2276","reported_seq":"2334","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:53.309081+0000","last_change":"2020-11-16T22:36:53.309146+0000","last_active":"2020-11-16T22:36:53.309081+0000","last_peered":"2020-11-16T22:36:53.309081+0000","last_clean":"2020-11-16T22:36:53.309081+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:53.309081+0000","last_undegraded":"2020-11-16T22:36:53.309081+0000","last_fullsized":"2020-11-16T22:36:53.309081+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2276","last_scrub_stamp":"2020-11-16T22:36:53.309077+0000","last_deep_scrub":"223'2276","last_deep_scrub_stamp":"2020-11-16T22:36:53.309077+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.309077+0000","log_size":2276,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27574","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:06.027772+0000","last_change":"2020-11-16T22:37:06.027875+0000","last_active":"2020-11-16T22:37:06.027772+0000","last_peered":"2020-11-16T22:37:06.027772+0000","last_clean":"2020-11-16T22:37:06.027772+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:37:06.027772+0000","last_undegraded":"2020-11-16T22:37:06.027772+0000","last_fullsized":"2020-11-16T22:37:06.027772+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'13357","last_scrub_stamp":"2020-11-16T22:37:06.027764+0000","last_deep_scrub":"204'13357","last_deep_scrub_stamp":"2020-11-16T22:37:06.027764+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.027764+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"223'2184","reported_seq":"2313","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:05.101150+0000","last_change":"2020-11-16T22:37:05.101215+0000","last_active":"2020-11-16T22:37:05.101150+0000","last_peered":"2020-11-16T22:37:05.101150+0000","last_clean":"2020-11-16T22:37:05.101150+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:37:05.101150+0000","last_undegraded":"2020-11-16T22:37:05.101150+0000","last_fullsized":"2020-11-16T22:37:05.101150+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2184","last_scrub_stamp":"2020-11-16T22:37:05.101144+0000","last_deep_scrub":"223'2184","last_deep_scrub_stamp":"2020-11-16T22:37:05.101144+0000","last_clean_scrub_stamp":"2020-11-16T22:37:05.101144+0000","log_size":2184,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"107","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:39.333478+0000","last_change":"2020-11-16T22:36:39.333552+0000","last_active":"2020-11-16T22:36:39.333478+0000","last_peered":"2020-11-16T22:36:39.333478+0000","last_clean":"2020-11-16T22:36:39.333478+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:39.333478+0000","last_undegraded":"2020-11-16T22:36:39.333478+0000","last_fullsized":"2020-11-16T22:36:39.333478+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_clean_scrub_stamp":"2020-11-16T22:36:39.333472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24226","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.269459+0000","last_change":"2020-11-16T22:36:33.269536+0000","last_active":"2020-11-16T22:36:33.269459+0000","last_peered":"2020-11-16T22:36:33.269459+0000","last_clean":"2020-11-16T22:36:33.269459+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:33.269459+0000","last_undegraded":"2020-11-16T22:36:33.269459+0000","last_fullsized":"2020-11-16T22:36:33.269459+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'11769","last_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_deep_scrub":"204'11769","last_deep_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.269452+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:44.000800+0000","last_change":"2020-11-16T22:36:44.000867+0000","last_active":"2020-11-16T22:36:44.000800+0000","last_peered":"2020-11-16T22:36:44.000800+0000","last_clean":"2020-11-16T22:36:44.000800+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:44.000800+0000","last_undegraded":"2020-11-16T22:36:44.000800+0000","last_fullsized":"2020-11-16T22:36:44.000800+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_clean_scrub_stamp":"2020-11-16T22:36:44.000794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"223'2294","reported_seq":"2383","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:03.072932+0000","last_change":"2020-11-16T22:37:03.073003+0000","last_active":"2020-11-16T22:37:03.072932+0000","last_peered":"2020-11-16T22:37:03.072932+0000","last_clean":"2020-11-16T22:37:03.072932+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:37:03.072932+0000","last_undegraded":"2020-11-16T22:37:03.072932+0000","last_fullsized":"2020-11-16T22:37:03.072932+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2294","last_scrub_stamp":"2020-11-16T22:37:03.072926+0000","last_deep_scrub":"223'2294","last_deep_scrub_stamp":"2020-11-16T22:37:03.072926+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.072926+0000","log_size":2294,"ondisk_log_size":2294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34463,"ondisk_log_size":34463,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468056,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:48 2020","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Nov 16 22:37:04 2020","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.68899999999999995,"15min":0.61299999999999999},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.375,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.66900000000000004,"15min":0.58499999999999996},"min":{"1min":0.627,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.872}]}]},{"osd":0,"up_from":205,"seq":880468295698,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79184,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31589,"kb_avail":94292656,"statfs":{"total":96636764160,"available":96555679744,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32347724},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:36:49 2020","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Nov 16 22:36:49 2020","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.111},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":1.125}]}]},{"osd":1,"up_from":8,"seq":34359738539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356596,"kb_used_data":54700,"kb_used_omap":0,"kb_used_meta":301888,"kb_avail":94015244,"statfs":{"total":96636764160,"available":96271609856,"internally_reserved":0,"allocated":56012800,"data_stored":63879065,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309133312},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:36:49 2020","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Nov 16 22:37:08 2020","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.61399999999999999,"15min":0.56799999999999995},"min":{"1min":0.63400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.67600000000000005,"15min":0.58499999999999996},"min":{"1min":0.59299999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.73999999999999999}]}]}]}} 2020-11-16T22:37:13.083 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2020-11-16T22:23:17.119717+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=23, tm_sec=17, tm_wday=0, tm_yday=321, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=16, tm_hour=22, tm_min=36, tm_sec=26, tm_wday=0, tm_yday=321, tm_isdst=0) 2020-11-16T22:37:13.286 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-16T22:37:33.288 INFO:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:38:03.173 INFO:teuthology.orchestra.run.smithi138.stdout: 2020-11-16T22:38:03.175 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2020-11-16T22:38:03.185 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":666,"stamp":"2020-11-16T22:38:01.821985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113220706,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":56885,"num_read_kb":1856191,"num_write":121480,"num_write_kb":1051089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7246,"num_bytes_recovered":151159831,"num_keys_recovered":232,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21239,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100587,"ondisk_log_size":100587,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":707956,"kb_used_data":151580,"kb_used_omap":117,"kb_used_meta":556234,"kb_avail":282407564,"statfs":{"total":289910292480,"available":289185345536,"internally_reserved":0,"allocated":155217920,"data_stored":229330695,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":119921,"internal_metadata":569584527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002762"},"pg_stats":[{"pgid":"4.2","version":"204'3435","reported_seq":"5211","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:38.222678+0000","last_change":"2020-11-16T22:36:38.222761+0000","last_active":"2020-11-16T22:36:38.222678+0000","last_peered":"2020-11-16T22:36:38.222678+0000","last_clean":"2020-11-16T22:36:38.222678+0000","last_became_active":"2020-11-16T22:35:51.015654+0000","last_became_peered":"2020-11-16T22:35:51.015654+0000","last_unstale":"2020-11-16T22:36:38.222678+0000","last_undegraded":"2020-11-16T22:36:38.222678+0000","last_fullsized":"2020-11-16T22:36:38.222678+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'3435","last_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_deep_scrub":"204'3435","last_deep_scrub_stamp":"2020-11-16T22:36:38.222672+0000","last_clean_scrub_stamp":"2020-11-16T22:36:38.222672+0000","log_size":3435,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1582,"num_read_kb":67694,"num_write":3505,"num_write_kb":32974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":6677010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:36.268887+0000","last_change":"2020-11-16T22:36:36.268958+0000","last_active":"2020-11-16T22:36:36.268887+0000","last_peered":"2020-11-16T22:36:36.268887+0000","last_clean":"2020-11-16T22:36:36.268887+0000","last_became_active":"2020-11-16T22:36:21.177263+0000","last_became_peered":"2020-11-16T22:36:21.177263+0000","last_unstale":"2020-11-16T22:36:36.268887+0000","last_undegraded":"2020-11-16T22:36:36.268887+0000","last_fullsized":"2020-11-16T22:36:36.268887+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:36.268882+0000","last_clean_scrub_stamp":"2020-11-16T22:36:36.268882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"223'5379","reported_seq":"5606","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:51.204392+0000","last_change":"2020-11-16T22:36:51.204461+0000","last_active":"2020-11-16T22:36:51.204392+0000","last_peered":"2020-11-16T22:36:51.204392+0000","last_clean":"2020-11-16T22:36:51.204392+0000","last_became_active":"2020-11-16T22:35:52.020575+0000","last_became_peered":"2020-11-16T22:35:52.020575+0000","last_unstale":"2020-11-16T22:36:51.204392+0000","last_undegraded":"2020-11-16T22:36:51.204392+0000","last_fullsized":"2020-11-16T22:36:51.204392+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'5379","last_scrub_stamp":"2020-11-16T22:36:51.204386+0000","last_deep_scrub":"223'5379","last_deep_scrub_stamp":"2020-11-16T22:36:51.204386+0000","last_clean_scrub_stamp":"2020-11-16T22:36:51.204386+0000","log_size":5379,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777376,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":24688771,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"204'9703","reported_seq":"17744","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:07.053747+0000","last_change":"2020-11-16T22:37:07.053819+0000","last_active":"2020-11-16T22:37:07.053747+0000","last_peered":"2020-11-16T22:37:07.053747+0000","last_clean":"2020-11-16T22:37:07.053747+0000","last_became_active":"2020-11-16T22:36:21.177843+0000","last_became_peered":"2020-11-16T22:36:21.177843+0000","last_unstale":"2020-11-16T22:37:07.053747+0000","last_undegraded":"2020-11-16T22:37:07.053747+0000","last_fullsized":"2020-11-16T22:37:07.053747+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'9703","last_scrub_stamp":"2020-11-16T22:37:07.053743+0000","last_deep_scrub":"204'9703","last_deep_scrub_stamp":"2020-11-16T22:37:07.053743+0000","last_clean_scrub_stamp":"2020-11-16T22:37:07.053743+0000","log_size":9703,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5135,"num_read_kb":172838,"num_write":9487,"num_write_kb":67857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1366,"num_bytes_recovered":3752854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"163","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:34.023339+0000","last_change":"2020-11-16T22:36:34.023420+0000","last_active":"2020-11-16T22:36:34.023339+0000","last_peered":"2020-11-16T22:36:34.023339+0000","last_clean":"2020-11-16T22:36:34.023339+0000","last_became_active":"2020-11-16T22:35:48.982393+0000","last_became_peered":"2020-11-16T22:35:48.982393+0000","last_unstale":"2020-11-16T22:36:34.023339+0000","last_undegraded":"2020-11-16T22:36:34.023339+0000","last_fullsized":"2020-11-16T22:36:34.023339+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:34.023333+0000","last_clean_scrub_stamp":"2020-11-16T22:36:34.023333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"223'4380","reported_seq":"4598","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:46.029417+0000","last_change":"2020-11-16T22:36:46.029476+0000","last_active":"2020-11-16T22:36:46.029417+0000","last_peered":"2020-11-16T22:36:46.029417+0000","last_clean":"2020-11-16T22:36:46.029417+0000","last_became_active":"2020-11-16T22:35:49.029797+0000","last_became_peered":"2020-11-16T22:35:49.029797+0000","last_unstale":"2020-11-16T22:36:46.029417+0000","last_undegraded":"2020-11-16T22:36:46.029417+0000","last_fullsized":"2020-11-16T22:36:46.029417+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'4380","last_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_deep_scrub":"223'4380","last_deep_scrub_stamp":"2020-11-16T22:36:46.029411+0000","last_clean_scrub_stamp":"2020-11-16T22:36:46.029411+0000","log_size":4380,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971554,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":21051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":37748804,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"204'4629","reported_seq":"6910","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:10.121465+0000","last_change":"2020-11-16T22:37:10.121533+0000","last_active":"2020-11-16T22:37:10.121465+0000","last_peered":"2020-11-16T22:37:10.121465+0000","last_clean":"2020-11-16T22:37:10.121465+0000","last_became_active":"2020-11-16T22:36:21.178085+0000","last_became_peered":"2020-11-16T22:36:21.178085+0000","last_unstale":"2020-11-16T22:37:10.121465+0000","last_undegraded":"2020-11-16T22:37:10.121465+0000","last_fullsized":"2020-11-16T22:37:10.121465+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'4629","last_scrub_stamp":"2020-11-16T22:37:10.121459+0000","last_deep_scrub":"204'4629","last_deep_scrub_stamp":"2020-11-16T22:37:10.121459+0000","last_clean_scrub_stamp":"2020-11-16T22:37:10.121459+0000","log_size":4629,"ondisk_log_size":4629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2043,"num_read_kb":79341,"num_write":4651,"num_write_kb":30645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":1819335,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"144","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:35.229186+0000","last_change":"2020-11-16T22:36:35.229272+0000","last_active":"2020-11-16T22:36:35.229186+0000","last_peered":"2020-11-16T22:36:35.229186+0000","last_clean":"2020-11-16T22:36:35.229186+0000","last_became_active":"2020-11-16T22:36:21.174337+0000","last_became_peered":"2020-11-16T22:36:21.174337+0000","last_unstale":"2020-11-16T22:36:35.229186+0000","last_undegraded":"2020-11-16T22:36:35.229186+0000","last_fullsized":"2020-11-16T22:36:35.229186+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:35.229180+0000","last_clean_scrub_stamp":"2020-11-16T22:36:35.229180+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"223'10835","reported_seq":"11016","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:30.347689+0000","last_change":"2020-11-16T22:36:30.347766+0000","last_active":"2020-11-16T22:36:30.347689+0000","last_peered":"2020-11-16T22:36:30.347689+0000","last_clean":"2020-11-16T22:36:30.347689+0000","last_became_active":"2020-11-16T22:36:17.159489+0000","last_became_peered":"2020-11-16T22:36:17.159489+0000","last_unstale":"2020-11-16T22:36:30.347689+0000","last_undegraded":"2020-11-16T22:36:30.347689+0000","last_fullsized":"2020-11-16T22:36:30.347689+0000","mapping_epoch":218,"log_start":"93'800","ondisk_log_start":"93'800","created":14,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'10835","last_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_deep_scrub":"223'10835","last_deep_scrub_stamp":"2020-11-16T22:36:30.347682+0000","last_clean_scrub_stamp":"2020-11-16T22:36:30.347682+0000","log_size":10035,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":21855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8536120,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"204'17258","reported_seq":"24002","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:52.129093+0000","last_change":"2020-11-16T22:36:52.129165+0000","last_active":"2020-11-16T22:36:52.129093+0000","last_peered":"2020-11-16T22:36:52.129093+0000","last_clean":"2020-11-16T22:36:52.129093+0000","last_became_active":"2020-11-16T22:30:08.059146+0000","last_became_peered":"2020-11-16T22:30:08.059146+0000","last_unstale":"2020-11-16T22:36:52.129093+0000","last_undegraded":"2020-11-16T22:36:52.129093+0000","last_fullsized":"2020-11-16T22:36:52.129093+0000","mapping_epoch":117,"log_start":"173'7200","ondisk_log_start":"173'7200","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'17258","last_scrub_stamp":"2020-11-16T22:36:52.129087+0000","last_deep_scrub":"204'17258","last_deep_scrub_stamp":"2020-11-16T22:36:52.129087+0000","last_clean_scrub_stamp":"2020-11-16T22:36:52.129087+0000","log_size":10058,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6534,"num_read_kb":133835,"num_write":17028,"num_write_kb":127372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2930685,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"186","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:37.968557+0000","last_change":"2020-11-16T22:36:37.968642+0000","last_active":"2020-11-16T22:36:37.968557+0000","last_peered":"2020-11-16T22:36:37.968557+0000","last_clean":"2020-11-16T22:36:37.968557+0000","last_became_active":"2020-11-16T22:35:49.999153+0000","last_became_peered":"2020-11-16T22:35:49.999153+0000","last_unstale":"2020-11-16T22:36:37.968557+0000","last_undegraded":"2020-11-16T22:36:37.968557+0000","last_fullsized":"2020-11-16T22:36:37.968557+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:37.968551+0000","last_clean_scrub_stamp":"2020-11-16T22:36:37.968551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"223'8","reported_seq":"176","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:32.029740+0000","last_change":"2020-11-16T22:36:32.029809+0000","last_active":"2020-11-16T22:36:32.029740+0000","last_peered":"2020-11-16T22:36:32.029740+0000","last_clean":"2020-11-16T22:36:32.029740+0000","last_became_active":"2020-11-16T22:35:50.031657+0000","last_became_peered":"2020-11-16T22:35:50.031657+0000","last_unstale":"2020-11-16T22:36:32.029740+0000","last_undegraded":"2020-11-16T22:36:32.029740+0000","last_fullsized":"2020-11-16T22:36:32.029740+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'8","last_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_deep_scrub":"223'8","last_deep_scrub_stamp":"2020-11-16T22:36:32.029734+0000","last_clean_scrub_stamp":"2020-11-16T22:36:32.029734+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"223'7796","reported_seq":"7990","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:50.343781+0000","last_change":"2020-11-16T22:36:50.343844+0000","last_active":"2020-11-16T22:36:50.343781+0000","last_peered":"2020-11-16T22:36:50.343781+0000","last_clean":"2020-11-16T22:36:50.343781+0000","last_became_active":"2020-11-16T22:36:21.176830+0000","last_became_peered":"2020-11-16T22:36:21.176830+0000","last_unstale":"2020-11-16T22:36:50.343781+0000","last_undegraded":"2020-11-16T22:36:50.343781+0000","last_fullsized":"2020-11-16T22:36:50.343781+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'7796","last_scrub_stamp":"2020-11-16T22:36:50.343775+0000","last_deep_scrub":"223'7796","last_deep_scrub_stamp":"2020-11-16T22:36:50.343775+0000","last_clean_scrub_stamp":"2020-11-16T22:36:50.343775+0000","log_size":7796,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":21478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":26640720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"204'4789","reported_seq":"9368","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:41.287297+0000","last_change":"2020-11-16T22:36:41.287378+0000","last_active":"2020-11-16T22:36:41.287297+0000","last_peered":"2020-11-16T22:36:41.287297+0000","last_clean":"2020-11-16T22:36:41.287297+0000","last_became_active":"2020-11-16T22:35:55.991032+0000","last_became_peered":"2020-11-16T22:35:55.991032+0000","last_unstale":"2020-11-16T22:36:41.287297+0000","last_undegraded":"2020-11-16T22:36:41.287297+0000","last_fullsized":"2020-11-16T22:36:41.287297+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'4789","last_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_deep_scrub":"204'4789","last_deep_scrub_stamp":"2020-11-16T22:36:41.287291+0000","last_clean_scrub_stamp":"2020-11-16T22:36:41.287291+0000","log_size":4789,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2137,"num_read_kb":66234,"num_write":4748,"num_write_kb":29935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":1875424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.056454+0000","last_change":"2020-11-16T22:36:33.056522+0000","last_active":"2020-11-16T22:36:33.056454+0000","last_peered":"2020-11-16T22:36:33.056454+0000","last_clean":"2020-11-16T22:36:33.056454+0000","last_became_active":"2020-11-16T22:35:50.031639+0000","last_became_peered":"2020-11-16T22:35:50.031639+0000","last_unstale":"2020-11-16T22:36:33.056454+0000","last_undegraded":"2020-11-16T22:36:33.056454+0000","last_fullsized":"2020-11-16T22:36:33.056454+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:33.056448+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.056448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"223'119","reported_seq":"355","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:48.259053+0000","last_change":"2020-11-16T22:36:48.259135+0000","last_active":"2020-11-16T22:36:48.259053+0000","last_peered":"2020-11-16T22:36:48.259053+0000","last_clean":"2020-11-16T22:36:48.259053+0000","last_became_active":"2020-11-16T22:36:21.173256+0000","last_became_peered":"2020-11-16T22:36:21.173256+0000","last_unstale":"2020-11-16T22:36:48.259053+0000","last_undegraded":"2020-11-16T22:36:48.259053+0000","last_fullsized":"2020-11-16T22:36:48.259053+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'119","last_scrub_stamp":"2020-11-16T22:36:48.259047+0000","last_deep_scrub":"223'119","last_deep_scrub_stamp":"2020-11-16T22:36:48.259047+0000","last_clean_scrub_stamp":"2020-11-16T22:36:48.259047+0000","log_size":119,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4167390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":111,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"204'16676","reported_seq":"33970","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:40.276217+0000","last_change":"2020-11-16T22:36:40.276311+0000","last_active":"2020-11-16T22:36:40.276217+0000","last_peered":"2020-11-16T22:36:40.276217+0000","last_clean":"2020-11-16T22:36:40.276217+0000","last_became_active":"2020-11-16T22:30:51.067762+0000","last_became_peered":"2020-11-16T22:30:51.067762+0000","last_unstale":"2020-11-16T22:36:40.276217+0000","last_undegraded":"2020-11-16T22:36:40.276217+0000","last_fullsized":"2020-11-16T22:36:40.276217+0000","mapping_epoch":137,"log_start":"98'6600","ondisk_log_start":"98'6600","created":15,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'16676","last_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_deep_scrub":"204'16676","last_deep_scrub_stamp":"2020-11-16T22:36:40.276200+0000","last_clean_scrub_stamp":"2020-11-16T22:36:40.276200+0000","log_size":10076,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16150,"num_read_kb":399765,"num_write":20440,"num_write_kb":320462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":7380764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"116","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:29.201097+0000","last_change":"2020-11-16T22:36:29.201173+0000","last_active":"2020-11-16T22:36:29.201097+0000","last_peered":"2020-11-16T22:36:29.201097+0000","last_clean":"2020-11-16T22:36:29.201097+0000","last_became_active":"2020-11-16T22:35:49.993569+0000","last_became_peered":"2020-11-16T22:35:49.993569+0000","last_unstale":"2020-11-16T22:36:29.201097+0000","last_undegraded":"2020-11-16T22:36:29.201097+0000","last_fullsized":"2020-11-16T22:36:29.201097+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:29.201091+0000","last_clean_scrub_stamp":"2020-11-16T22:36:29.201091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"223'2276","reported_seq":"2334","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:53.309081+0000","last_change":"2020-11-16T22:36:53.309146+0000","last_active":"2020-11-16T22:36:53.309081+0000","last_peered":"2020-11-16T22:36:53.309081+0000","last_clean":"2020-11-16T22:36:53.309081+0000","last_became_active":"2020-11-16T22:36:21.177066+0000","last_became_peered":"2020-11-16T22:36:21.177066+0000","last_unstale":"2020-11-16T22:36:53.309081+0000","last_undegraded":"2020-11-16T22:36:53.309081+0000","last_fullsized":"2020-11-16T22:36:53.309081+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2276","last_scrub_stamp":"2020-11-16T22:36:53.309077+0000","last_deep_scrub":"223'2276","last_deep_scrub_stamp":"2020-11-16T22:36:53.309077+0000","last_clean_scrub_stamp":"2020-11-16T22:36:53.309077+0000","log_size":2276,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12583555,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":12626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8389894,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"204'13357","reported_seq":"27574","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:06.027772+0000","last_change":"2020-11-16T22:37:06.027875+0000","last_active":"2020-11-16T22:37:06.027772+0000","last_peered":"2020-11-16T22:37:06.027772+0000","last_clean":"2020-11-16T22:37:06.027772+0000","last_became_active":"2020-11-16T22:36:21.175653+0000","last_became_peered":"2020-11-16T22:36:21.175653+0000","last_unstale":"2020-11-16T22:37:06.027772+0000","last_undegraded":"2020-11-16T22:37:06.027772+0000","last_fullsized":"2020-11-16T22:37:06.027772+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'13357","last_scrub_stamp":"2020-11-16T22:37:06.027764+0000","last_deep_scrub":"204'13357","last_deep_scrub_stamp":"2020-11-16T22:37:06.027764+0000","last_clean_scrub_stamp":"2020-11-16T22:37:06.027764+0000","log_size":13357,"ondisk_log_size":13357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11246,"num_read_kb":426354,"num_write":13691,"num_write_kb":159033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1434,"num_bytes_recovered":5800200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"223'2184","reported_seq":"2313","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:05.101150+0000","last_change":"2020-11-16T22:37:05.101215+0000","last_active":"2020-11-16T22:37:05.101150+0000","last_peered":"2020-11-16T22:37:05.101150+0000","last_clean":"2020-11-16T22:37:05.101150+0000","last_became_active":"2020-11-16T22:36:21.177624+0000","last_became_peered":"2020-11-16T22:36:21.177624+0000","last_unstale":"2020-11-16T22:37:05.101150+0000","last_undegraded":"2020-11-16T22:37:05.101150+0000","last_fullsized":"2020-11-16T22:37:05.101150+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2184","last_scrub_stamp":"2020-11-16T22:37:05.101144+0000","last_deep_scrub":"223'2184","last_deep_scrub_stamp":"2020-11-16T22:37:05.101144+0000","last_clean_scrub_stamp":"2020-11-16T22:37:05.101144+0000","log_size":2184,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8389183,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2164,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"107","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:39.333478+0000","last_change":"2020-11-16T22:36:39.333552+0000","last_active":"2020-11-16T22:36:39.333478+0000","last_peered":"2020-11-16T22:36:39.333478+0000","last_clean":"2020-11-16T22:36:39.333478+0000","last_became_active":"2020-11-16T22:36:21.177175+0000","last_became_peered":"2020-11-16T22:36:21.177175+0000","last_unstale":"2020-11-16T22:36:39.333478+0000","last_undegraded":"2020-11-16T22:36:39.333478+0000","last_fullsized":"2020-11-16T22:36:39.333478+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:39.333472+0000","last_clean_scrub_stamp":"2020-11-16T22:36:39.333472+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"204'11769","reported_seq":"24226","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:33.269459+0000","last_change":"2020-11-16T22:36:33.269536+0000","last_active":"2020-11-16T22:36:33.269459+0000","last_peered":"2020-11-16T22:36:33.269459+0000","last_clean":"2020-11-16T22:36:33.269459+0000","last_became_active":"2020-11-16T22:35:51.013181+0000","last_became_peered":"2020-11-16T22:35:51.013181+0000","last_unstale":"2020-11-16T22:36:33.269459+0000","last_undegraded":"2020-11-16T22:36:33.269459+0000","last_fullsized":"2020-11-16T22:36:33.269459+0000","mapping_epoch":207,"log_start":"128'1700","ondisk_log_start":"128'1700","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'11769","last_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_deep_scrub":"204'11769","last_deep_scrub_stamp":"2020-11-16T22:36:33.269452+0000","last_clean_scrub_stamp":"2020-11-16T22:36:33.269452+0000","log_size":10069,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12029,"num_read_kb":509539,"num_write":12795,"num_write_kb":167247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":2336338,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"168","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:36:44.000800+0000","last_change":"2020-11-16T22:36:44.000867+0000","last_active":"2020-11-16T22:36:44.000800+0000","last_peered":"2020-11-16T22:36:44.000800+0000","last_clean":"2020-11-16T22:36:44.000800+0000","last_became_active":"2020-11-16T22:35:50.116925+0000","last_became_peered":"2020-11-16T22:35:50.116925+0000","last_unstale":"2020-11-16T22:36:44.000800+0000","last_undegraded":"2020-11-16T22:36:44.000800+0000","last_fullsized":"2020-11-16T22:36:44.000800+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:36:44.000794+0000","last_clean_scrub_stamp":"2020-11-16T22:36:44.000794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"223'2294","reported_seq":"2383","reported_epoch":"223","state":"active+clean","last_fresh":"2020-11-16T22:37:03.072932+0000","last_change":"2020-11-16T22:37:03.073003+0000","last_active":"2020-11-16T22:37:03.072932+0000","last_peered":"2020-11-16T22:37:03.072932+0000","last_clean":"2020-11-16T22:37:03.072932+0000","last_became_active":"2020-11-16T22:36:21.175842+0000","last_became_peered":"2020-11-16T22:36:21.175842+0000","last_unstale":"2020-11-16T22:37:03.072932+0000","last_undegraded":"2020-11-16T22:37:03.072932+0000","last_fullsized":"2020-11-16T22:37:03.072932+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'2294","last_scrub_stamp":"2020-11-16T22:37:03.072926+0000","last_deep_scrub":"223'2294","last_deep_scrub_stamp":"2020-11-16T22:37:03.072926+0000","last_clean_scrub_stamp":"2020-11-16T22:37:03.072926+0000","log_size":2294,"ondisk_log_size":2294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":8780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12582912,"num_keys_recovered":212,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56856,"num_read_kb":1855600,"num_write":86345,"num_write_kb":935525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7185,"num_bytes_recovered":32572610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66116,"ondisk_log_size":66116,"up":16,"acting":16,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":113220706,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":29,"num_read_kb":591,"num_write":35131,"num_write_kb":115544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":118587221,"num_keys_recovered":232,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4710,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148647936,"data_stored":226441412,"data_compressed":19654874,"data_compressed_allocated":77856768,"data_compressed_original":155713536,"omap_allocated":103130,"internal_metadata":0},"log_size":34463,"ondisk_log_size":34463,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16529,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12579,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":108,"seq":463856468066,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272176,"kb_used_data":49384,"kb_used_omap":26,"kb_used_meta":222757,"kb_avail":94099664,"statfs":{"total":96636764160,"available":96358055936,"internally_reserved":0,"allocated":50569216,"data_stored":72239735,"data_compressed":5608601,"data_compressed_allocated":22913024,"data_compressed_original":45826048,"omap_allocated":27325,"internal_metadata":228103491},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:53 2020","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.495,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.90100000000000002}]},{"osd":1,"last update":"Mon Nov 16 22:37:04 2020","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.68899999999999995,"15min":0.61299999999999999},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.375,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.66900000000000004,"15min":0.58499999999999996},"min":{"1min":0.627,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.81299999999999994}]}]},{"osd":0,"up_from":205,"seq":880468295708,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79184,"kb_used_data":47496,"kb_used_omap":90,"kb_used_meta":31589,"kb_avail":94292656,"statfs":{"total":96636764160,"available":96555679744,"internally_reserved":0,"allocated":48635904,"data_stored":93211895,"data_compressed":11483918,"data_compressed_allocated":45834240,"data_compressed_original":91668480,"omap_allocated":92596,"internal_metadata":32347724},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 16 22:37:54 2020","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.54200000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.85399999999999998}]},{"osd":2,"last update":"Mon Nov 16 22:37:54 2020","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.78600000000000003,"15min":0.76500000000000001},"min":{"1min":0.623,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.64500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.79900000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356596,"kb_used_data":54700,"kb_used_omap":0,"kb_used_meta":301888,"kb_avail":94015244,"statfs":{"total":96636764160,"available":96271609856,"internally_reserved":0,"allocated":56012800,"data_stored":63879065,"data_compressed":2562355,"data_compressed_allocated":9109504,"data_compressed_original":18219008,"omap_allocated":0,"internal_metadata":309133312},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 16 22:37:50 2020","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.56799999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Nov 16 22:37:08 2020","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.61399999999999999,"15min":0.56799999999999995},"min":{"1min":0.63400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.67600000000000005,"15min":0.58499999999999996},"min":{"1min":0.59299999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1639999999999999,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":0.78800000000000003}]}]}]}} 2020-11-16T22:38:04.208 INFO:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-11-16T22:38:04.528 INFO:teuthology.misc:Shutting down mds daemons... 2020-11-16T22:38:04.528 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:38:04.529 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.538 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:38:04.538 INFO:teuthology.misc:Shutting down osd daemons... 2020-11-16T22:38:04.539 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-11-16T22:38:04.539 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.579 INFO:tasks.ceph.osd.0:Stopped 2020-11-16T22:38:04.579 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-11-16T22:38:04.579 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.646 INFO:tasks.ceph.osd.1:Stopped 2020-11-16T22:38:04.646 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-11-16T22:38:04.647 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.700 INFO:tasks.ceph.osd.2:Stopped 2020-11-16T22:38:04.700 INFO:teuthology.misc:Shutting down mgr daemons... 2020-11-16T22:38:04.700 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-11-16T22:38:04.701 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.718 INFO:tasks.ceph.mgr.x:Stopped 2020-11-16T22:38:04.718 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2020-11-16T22:38:04.718 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.775 INFO:tasks.ceph.mgr.y:Stopped 2020-11-16T22:38:04.776 INFO:teuthology.misc:Shutting down mon daemons... 2020-11-16T22:38:04.776 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-16T22:38:04.776 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.826 INFO:tasks.ceph.mon.a:Stopped 2020-11-16T22:38:04.826 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-16T22:38:04.826 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.876 INFO:tasks.ceph.mon.b:Stopped 2020-11-16T22:38:04.876 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-16T22:38:04.876 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:38:04.925 INFO:tasks.ceph.mon.c:Stopped 2020-11-16T22:38:04.926 INFO:tasks.ceph:Checking cluster log for badness... 2020-11-16T22:38:04.926 INFO:teuthology.orchestra.run.smithi138:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v '\(MDS_TRIM\)' | egrep -v '\(MDS_SLOW_REQUEST\)' | egrep -v MDS_SLOW_METADATA_IO | egrep -v 'Behind on trimming' | head -n 1 2020-11-16T22:38:04.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-16T22:38:04.997 INFO:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-11-16T22:38:05.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi194.front.sepia.ceph.com 2020-11-16T22:38:05.102 INFO:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-11-16T22:38:05.236 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi122.front.sepia.ceph.com 2020-11-16T22:38:05.236 INFO:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-11-16T22:38:05.380 INFO:tasks.ceph:Archiving mon data... 2020-11-16T22:38:05.381 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/data/mon.a.tgz 2020-11-16T22:38:05.382 INFO:teuthology.orchestra.run.smithi138:> mktemp 2020-11-16T22:38:05.396 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.AGx4lyQJE2 2020-11-16T22:38:05.397 INFO:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AGx4lyQJE2 2020-11-16T22:38:05.823 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.AGx4lyQJE2 2020-11-16T22:38:05.908 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.AGx4lyQJE2 is 2MB 2020-11-16T22:38:31.290 INFO:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.AGx4lyQJE2 2020-11-16T22:38:31.308 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/data/mon.b.tgz 2020-11-16T22:38:31.308 INFO:teuthology.orchestra.run.smithi138:> mktemp 2020-11-16T22:38:31.364 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.wbeBOs0UDF 2020-11-16T22:38:31.364 INFO:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wbeBOs0UDF 2020-11-16T22:38:31.796 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.wbeBOs0UDF 2020-11-16T22:38:31.880 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.wbeBOs0UDF is 2MB 2020-11-16T22:39:02.794 INFO:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.wbeBOs0UDF 2020-11-16T22:39:03.015 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/data/mon.c.tgz 2020-11-16T22:39:03.015 INFO:teuthology.orchestra.run.smithi138:> mktemp 2020-11-16T22:39:03.070 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.gDruK4ZSph 2020-11-16T22:39:03.071 INFO:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gDruK4ZSph 2020-11-16T22:39:03.505 INFO:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.gDruK4ZSph 2020-11-16T22:39:03.589 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.gDruK4ZSph is 2MB 2020-11-16T22:39:24.718 INFO:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.gDruK4ZSph 2020-11-16T22:39:24.736 INFO:tasks.ceph:Cleaning ceph cluster... 2020-11-16T22:39:24.737 INFO:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:39:24.739 INFO:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:39:24.779 INFO:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:39:24.781 INFO:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-16T22:39:24.824 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-11-16T22:39:24.824 INFO:teuthology.orchestra.run.smithi138:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:39:24.850 INFO:teuthology.orchestra.run.smithi173:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:39:24.852 INFO:teuthology.orchestra.run.smithi194:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:39:24.855 INFO:teuthology.orchestra.run.smithi122:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-16T22:39:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:39:24.883 INFO:teuthology.orchestra.run.smithi194.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:39:24.889 INFO:teuthology.orchestra.run.smithi173.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:39:24.897 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-16T22:39:24.903 INFO:tasks.ceph:Archiving crash dumps... 2020-11-16T22:39:24.905 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/remote/smithi138/crash 2020-11-16T22:39:24.906 INFO:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:39:24.956 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/remote/smithi173/crash 2020-11-16T22:39:24.957 INFO:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:39:25.003 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/remote/smithi194/crash 2020-11-16T22:39:25.004 INFO:teuthology.orchestra.run.smithi194:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:39:25.049 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/remote/smithi122/crash 2020-11-16T22:39:25.050 INFO:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-16T22:39:25.123 INFO:tasks.ceph:Compressing logs... 2020-11-16T22:39:25.124 INFO:teuthology.orchestra.run.smithi122:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:39:25.126 INFO:teuthology.orchestra.run.smithi138:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:39:25.332 INFO:teuthology.orchestra.run.smithi173:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:39:25.334 INFO:teuthology.orchestra.run.smithi194:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-16T22:44:09.944 INFO:tasks.ceph:Archiving logs... 2020-11-16T22:59:59.291 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-16_11:15:03-powercycle-master-distro-basic-smithi/5631024/remote/smithi138/log 2020-11-16T22:59:59.293 INFO:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/log/ceph -- .